Rebuild the streamable HTTP server transport around per-request dispatch

MA
4mo ago
+2481-1125

Rebuilds the legacy (2025-era, sessionful) streamable HTTP server transport around per-request
direct dispatch, deleting the fabricated duplex message pipe and the router that fanned it back out.

Motivation and Context

StreamableHTTPServerTransport used to reconstruct HTTP request boundaries on top of a
stdio-shaped seam: connect() yielded a (read_stream, write_stream) pair for a
JSONRPCDispatcher, and a message_router task read the session-wide write stream and fanned
messages back out into per-request queues (_request_streams), inferring "this POST is done"
from the response frame passing by. HTTP already knows request boundaries, so all of that
machinery (and its head-of-line and teardown races — #1363, #1764) was reconstructing
information the transport threw away. The 2026 route (_streamable_http_modern.py) already
dispatches each POST directly; this brings the legacy path onto the same shape while keeping
everything the 2025 spec needs (sessions, the GET stream, server-to-client requests, event-store
resumability, close_sse_stream() polling).

What changed:

  • RequestCorrelator (new shared/_correlation.py), extracted from JSONRPCDispatcher: the
    outbound pending table + the whole send/await/abandon policy, the inbound in-flight table +
    peer-cancel, and the exception-to-wire boundary for inbound handlers now live in one place.
    JSONRPCDispatcher delegates to it (behaviour unchanged); the HTTP transport shares it. The
    cancelled-request answer policy lives at exactly one site in it, which #3188 will need to
    target when it lands (a suppressed answer will need the response stream terminated another
    way — noted in a comment there).
  • _MessageChannel: one per response stream. write() is store-first (event store) then
    forward to the attached SSE response, serialized per channel so wire order always matches
    store order; it returns whether the message reached the store or the response, and never
    raises. The standalone GET stream is the same object. Attach/detach is the resumability
    primitive (close_sse_stream() detaches; a Last-Event-ID reconnect replays then
    re-attaches); attach() returns None on a channel whose life is over.
  • StreamableHTTPServerTransport is now the per-session core: session id, Connection, one
    ServerRunner, the correlator, the open channels, and a session task hosting request-handler
    tasks (so a dropped connection does not cancel a request — the 2025 spec's disconnect ≠ cancel
    rule). Each POSTed request is dispatched to the handler kernel directly; a POSTed
    response resolves the waiting server-to-client request; a POSTed notification is handled after
    the 202. Requests that arrive while an initialize is still being served wait for the
    handshake to commit (the ordering the stream-pair driver's parked read loop used to provide).
    connect() and the fabricated stream pair are gone — the one API removal, and nothing in
    docs/examples used it (see the migration note).
  • StreamableHTTPSessionManager: identical public surface; it binds a transport to the
    Server per session and hosts the session task instead of a serve_loop.

Behaviour clarified in the same change (all in docs/migration.md):

  • In JSON-response mode a request-scoped server-to-client request raises
    NoBackChannelError instead of hanging (connection-scoped sends still ride the GET stream).
  • A server-to-client request that can reach no client at all (no attached stream and nothing
    storing it) fails the handler with CONNECTION_CLOSED instead of parking for an answer that
    cannot arrive.
  • A failing EventStore.store_event degrades resumability for that message rather than taking
    the stream down: it is delivered live and the store's exception is logged, never on the wire.
  • Stream ids handed to the EventStore are minted by the transport in a session-scoped
    namespace (opaque), so a client-chosen request id can no longer name the standalone GET
    stream and two sessions sharing a store can no longer replay each other's frames.
  • A Last-Event-ID GET on a server with no event store behaves as a plain GET instead of
    returning no response; two concurrent POSTs sharing a JSON-RPC id each keep their own response
    stream instead of the second silently taking over the first's queue.

How Has This Been Tested?

  • tests/interaction/ (the interaction-model suite pinning observable behaviour) passes unchanged
    — 867 tests, across the in-memory / streamable-http / streamable-http-stateless / sse matrix —
    as does tests/examples/ (the story legs over HTTP) and the full suite (5523 tests) at 100%
    branch coverage with strict-no-cover clean.
  • The conformance harness against mcp-everything-server locally: the active suite is 42/42,
    and the full --suite all run passes everything except the already-baselined tasks-*
    scenarios ("baseline check passed") — including server-sse-polling (priming-first,
    retry, Last-Event-ID replay), server-sse-multiple-streams, DNS-rebinding, and every
    sampling/elicitation/progress scenario, i.e. against the TypeScript SDK client on the real wire.
  • Driven by hand over a real socket (uvicorn): initialize (priming + session id), the GET
    stream (and 409 for a second one), tool calls over SSE, an overlapping re-initialize next to
    a tools/list on one session, a request id shaped like the internal GET-stream marker
    (served on its own stream), DELETE → 404-after-termination, plus the 400/404/405/409 probes.
  • Unit tests that reached into the removed router internals were triaged case by case: rewritten
    against the behaviour they guarded (priming-store failure → 500 with no leaked state, standalone
    teardown via close_standalone_sse_stream(), manager eviction of exited/crashed sessions,
    stateless cleanup, close-callback protocol-version gating) or deleted where the guarded race
    is now unrepresentable (#1764 router head-of-line, the standalone writer's between-dequeues
    window). New transport tests cover: terminating a session mid-request (SSE and JSON mode),
    client-posted progress for a server-initiated request, per-channel write ordering, a request
    suspended across a DELETE (refused, not run), a store failure costing only resumability (per
    request and on the standalone stream), an undeliverable server-to-client request failing fast,
    the init gate (including overlapping handshakes), shared-store session isolation, a request id
    colliding with the GET-stream marker, a replay resumed across termination, and concurrent
    POSTs sharing a request id.

Breaking Changes

StreamableHTTPServerTransport.connect() is removed; the transport is created and driven per
request by StreamableHTTPSessionManager (new keyword-only app / lifespan_state constructor
arguments). Code serving through streamable_http_app() / run(transport="streamable-http") or
mounting the session manager is unaffected; only hand-rolled async with transport.connect()
usage needs to move to the manager. Documented in docs/migration.md.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

The transport's public module surface (EventStore, EventMessage, EventCallback,
GET_STREAM_KEY, check_accept_headers, the header/pattern constants, close_sse_stream(),
close_standalone_sse_stream(), terminate(), handle_request()) and the manager's surface are
unchanged, and no interaction/story/conformance test needed touching. Recorded divergences that
this design would make trivial to change (accepting a second initialize on a session; an
unknown Last-Event-ID producing an empty stream) are deliberately left as they are so this stays
a behaviour-preserving change.

Follow-ups deliberately out of scope here (both pre-existing at the manager level): a session
that a client DELETEs stays registered until manager shutdown rather than being evicted (the
interaction suite currently pins that), and a session-id-less non-initialize POST still mints a
session before its 400.

Responses to the automated reviews are in the threads; the two rounds of findings that held up
(write ordering per channel, requests racing termination, event-store failures reaching the
wire or hanging a waiting handler, the client-namespace stream ids, and the SSE containment
drift) were fixed by making the underlying state unrepresentable rather than case by case.