Discover auth-server metadata before eager token refresh

Closed
MF
3mo ago
+126-7

On a cold start with a cached-but-expired token (reusing a stored refresh token before any 401), async_auth_flow refreshed before discovery ran, so oauth_metadata was None and _refresh_token fell back to {origin}/token. That drops any issuer path and 404s on servers whose token endpoint lives under a path (e.g. .../oauth2/api/v1/token) — the refresh fails, tokens are cleared, and the client is forced into interactive re-auth it may not be able to complete. Discover AS metadata before the eager refresh.

Adds _discover_oauth_metadata (pure discovery, driven through the auth flow) and _refresh_with_discovery, plus a regression test.

Fixes #3240

Motivation and Context

Hit this against a hosted MCP server whose authorization server isn't at the resource origin (token endpoint https://host/oauth2/api/v1/token, not https://host/token). A headless client with cached tokens disconnects every time the short-lived access token expires: on reconnect the eager refresh at the top of async_auth_flow runs before any 401/discovery, oauth_metadata is None, and _refresh_token uses urljoin(get_authorization_base_url(server_url), "/token") → {scheme}://{netloc}/token → 404 → _handle_refresh_response clears the tokens → the flow drops to interactive auth a background client can't complete.

The same path-stripping fallback exists in _get_token_endpoint, _perform_authorization_code_grant (/authorize) and DCR (/register), but those run inside the 401 branch after discovery, so metadata is already populated there — refresh is the one that fires before discovery, which is why it's the acute, silent case. This PR fixes the refresh path; the other fallbacks are left as-is.

How Has This Been Tested?

  • New regression test in tests/client/test_auth.py: cold start with an expired-but-refreshable token and no prior discovery, against an AS whose token_endpoint is under a path; asserts the refresh request targets the discovered endpoint, not {origin}/token.
  • uv run pytest tests/client/test_auth.py → 141 passed, 1 xfailed. uv run ruff check / ruff format --check clean.
  • Reproduced the original break and confirmed the fix end-to-end against a real hosted MCP server (Interactive Brokers) whose token endpoint lives under /oauth2/....

Breaking Changes

None. On a cold start with no cached metadata the eager refresh now issues discovery requests before the refresh; once metadata is known it's a no-op, so the fast path (already-valid or already-discovered) is unchanged.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

_discover_oauth_metadata is a pure-discovery async generator (PRM → AS metadata) that yields its requests through the outer httpx auth flow — noside-channel client — and only populates protected_resource_metadata / auth_server_url / oauth_metadata. _refresh_with_discovery wraps it + the existing refresh so async_auth_flow drives one sub-flow and stays under the module's complexity cap. It reuses the existing build_*_discovery_urls /handle_*_response / validate_metadata_issuer helpers, so discovery behaves exactly like the 401 path.