fix: maintain typed auth clients as remote source headers change
Summary
A credential source's typed auth clients, server links and copied credential headers now stay in line as the source and its servers are written, inside the same transaction as the write. Before this, only the organization-wide authclientbackfill run maintained them.
Covered writes: remoteMcp.createServerHeader, updateServerHeader, deleteServerHeader, remoteMcp.deleteServer, tunneledMcp.deleteServer, and the creation, backend switch and deletion of an MCP server.
Motivation
Two gaps, both leaving records that no longer match the configuration:
- A remote source's Authorization header got no typed client, no header copy and no server link until a run came along, and deleting the header left its client behind with no tombstone.
- The credential headers copied onto the auth clients of upstream OAuth and EMA clients were only maintained by the run. The serving path loads those copies and sends them with the upstream request, so a deleted header could still be sent, and a rotated one sent with its old value, until the next run.
Tunneled sources had no typed reconciliation at all: deleting one left its derived clients, links and copies live.
Impact
Correctness and freshness of the records the upstream serving path reads. No change to how a request is authorized or to which client serves it. Administrator-created clients, administrator detachments, credential choices and auth grants are untouched, and every path is scoped to the writing organization and project. A write whose evidence changes under it rolls back rather than committing records that do not match.
Technical details
The sync reuses the backfill's planner rather than restating it
The per-client copy decisions were extracted from copyOntoOAuthClients and copyOntoEMAClients into copyOntoOAuthClient and copyOntoEMAClient, which the run and the sync both call. ListOAuthClientSources gained an optional client-id filter so the sync reads one client's sources instead of the organization's. A test asserts that a backfill run straight after a sync reports no pending clients, links, copies, retirements or gaps.
A client is decided from all of its servers' sources
The rule that refuses a client shared across sources with credential headers reads every source its servers sit on, so the sync reads them all for the clients it re-decides — scoped to those clients, never the organization. A client left with no live server on any source carries no copies, so its own are retired.
A server that leaves every credential source
Switching a server to a toolset backend, or deleting a server that had none, leaves no source to re-decide its clients from. SyncServerDerivedCopies re-decides them from the server itself for those cases. It takes no lock: every copy write repeats the ownership rule it was decided from, so a concurrent change loses the write rather than overwriting it.
Tunneled sources
SyncTunneledSource mirrors SyncRemoteSource. No management API writes a tunneled source's headers and its deletion leaves them on the tombstoned row, so the source's liveness is what says they are gone. No tunneled header API was added.
Lock order is source-then-server everywhere
The sync writes to a source's mcp_servers rows and to their links, while a source's own writers lock the source first. The shared MCP server creation command, updateMcpServer and deleteMcpServer take the source lock up front and reject a concurrent backend switch as a retryable conflict.
The creation command also settles each source's liveness and project ownership under that lock, for both kinds. It was previously checked before any lock, so a deletion committing while the create waited could leave a server in front of a tombstoned source. LockRemoteSource and LockTunneledSource still accept a tombstoned source, because the update and delete paths have to lock one to retire its clients.
Association writes
Associating an upstream OAuth client with a server on a second source makes the client shared, and the ownership rule then says it carries neither source's credential headers — while the first source's copies are still on it. ListAuthClientHeaders loads those copies by auth client id and organization alone and revalidates no source ownership, and withoutDeniedSources only strips forwarded headers from refused request headers, so the first source's secret went to the second source's upstream until the next backfill run. Detaching the server whose source lent the headers leaves the mirror state.
All three writers of an association now re-decide the copies in their own transaction: the identity commit, the catalog identity-provider attachment, and the dashboard detach. The mechanism predates this PR, but this PR records copies at header-write time rather than only at backfill time, so the stale state was reachable in more windows than before.
Lock behaviour of the copies pass
It locks only the header copy rows it writes — no source, server, link or upstream client row — and every caller runs it after its own writes, so the copy rows are the last locks the transaction takes. Two orders needed correcting for that to hold: a source sync marks servers before writing copies, and MCP server deletion runs its syncs after the issuer cascade, which row-locks the remote_session_clients an identity commit writes before reaching the same copies.
Consistency under a concurrent source or header write comes from the writes rather than a lock: an added or updated copy repeats the ownership rule it was decided from, so an invalidating change makes the write affect no row; a retired copy is unconditional, so the decision to stop sending a credential always lands. A copy wrongly retired by a racing writer is restored by the next source write or backfill run.