#3663 feat(cas): shared HTTP and gRPC endpoint via per-request dispatch and a reverse proxy to the native transport

The CAS shared endpoint mirrors the merged control plane (spec D-020). One standard HTTP listener owns server.http.addr (HTTP/1.1, HTTP/2, optional TLS) and dispatches each request:

• Native gRPC (HTTP/2 + application/grpc*, excluding application/grpc-web*, which CAS does not serve) → an in-process httputil.ReverseProxy streaming to a private native gRPC server bound to an ephemeral loopback address — grpc.Server.Serve, the native transport, the same services and interceptors as the legacy listener, with the client's original :authority restored by the proxy's Rewrite.
• Everything else → the existing HTTP routes (downloads, status, CORS).

ByteStream keeps the native transport with no unbounded read-ahead in either direction: the proxy streams the request body into the private connection, whose HTTP/2 flow-control window paces the proxy's reads while the shared listener's window paces the client. This replaces the two earlier designs on this branch, each rejected by measurement: the per-request ServeHTTP adapter (~1.25 GiB live heap on a 1 GiB upload, −11.8% download) and cmux connection classification (native parity, but a silent matcher deadlocks grpc-go clients, the SETTINGS-writing matcher breaks plain-HTTP/2 connections with GOAWAY, and pooled mixed connections return 502 under a single-location ingress).

The private server is independent of the legacy listener: the legacy flag, address, and TLS settings are unchanged, and the shared endpoint works with the legacy listener disabled. Mixed native-gRPC and plain-HTTP requests share one upstream connection in any order — the control-plane property. The cmux dependency and its classification machinery are gone.

flowchart TD
    edge["Ingress: TLS termination, one / location, HTTP/2 (gRPC) upstream"] --> shared["CAS :8001: net/http, dispatch each request"]
    direct["VM / Compose: HTTP/1.1, h2c, or TLS"] --> shared
    shared -->|"application/grpc* + HTTP/2"| proxy["httputil.ReverseProxy (stdlib)"]
    proxy --> private["Private native gRPC server: 127.0.0.1 ephemeral"]
    shared -->|"ordinary HTTP"| routes["Existing HTTP routes"]
    legacy["Legacy listener :9001: unchanged flag, address, TLS"] --> services["Same services and interceptors"]
    private --> services

Helm chart

One Service port and one catch-all Ingress on the same hostname, matching the control-plane shape: path / with backend-protocol: GRPC (HTTP/2 upstream), grpc-services, and the upload body-size annotations; Traefik h2c Service annotation. Removed: the RPC-path second Ingress and its path list, the grpc-shared Service port alias, and the pool-isolation validations — per-request dispatch makes them unnecessary. cas.ingress.enabled defaults to false: the chart README now documents the enabling block. cas.ingressAPI, cas.serviceAPI, and the control-plane connection to the legacy CAS endpoint keep their existing settings.

Spec

• D-020 supersedes D-017 (cmux + SETTINGS contract), D-018 (path-split chart), and D-019 (connection-reuse isolation); the grpc-web exclusion is documented in the entry.
• R-010 supersedes R-009: native gRPC over HTTP/2 plus plain HTTP/1.1 and HTTP/2, mixed per request; ByteStream through the proxy.
• The merged control-plane design stays unchanged.

Measured evidence

1 GiB transport comparison, three runs per operation, disk-backed staging, reproduced with the benchmark command in app/artifact-cas/README.md (raw output in the PR discussion):

Listener	Upload	Download	Peak live heap (upload)
Legacy native :9001	233.92 MB/s	192.82 MB/s	29.13 MiB
Shared :8001 (proxy)	212.89 MB/s (−9.0%)	177.11 MB/s (−8.1%)	32.45 MiB

Within the R-003 10% shape with a thin margin; heap claims are scoped to the observed 1 GiB runs. TestCASSharedConcurrentUploadsMemoryBounded adds the concurrency dimension: four concurrent 128 MiB uploads through the dispatcher peaked at 58 MiB combined, so per-stream buffers do not accumulate toward the transferred bytes.

Backpressure is regression-tested in both directions through the production proxy (TestCASSharedPausedConsumerBackpressure, via a test-only pacing hook that is nil in production): a paused consumer stops the producer after bounded in-flight buffering — uploads and downloads alike — and cancellation releases both sides. This is the property whose absence disqualified the ServeHTTP adapter.

-race tests pass across the CAS: native gRPC over h2c and TLS, HTTP/1.1 and HTTP/2 requests, six mixed requests reusing one pooled connection, the auth/error matrix, mid-upload cancellation through the proxy, authority preservation through the proxy, private-socket cleanup on failed construction and before Start, and shutdown draining (HTTP first, then the private native server). A standalone probe (PR discussion) additionally covered gzip, binary metadata, trailers, rich error details, and downstream cancellation through the standard proxy. Chart-rendering tests assert the control-plane shape and unchanged legacy values.

Open before adoption

• Full R-002 authentication/authorization alignment matrix through the shared hostname (the native peer is now the proxy connection; authority is preserved and verified by test, but the full matrix remains).
• Original R-003 qualification against the current release with representative object storage, including multi-size heap measurements; the local 10%-gate margin is thin (−9.0% upload, −8.1% download).
• F5 and Traefik controller qualification, and a deployed ingress-nginx check of the single-location arrangement (the local kind harness was removed as out of scope; the reproduction lives in the discussion).
• Paused-consumer behavior through a real ingress, including nginx grpc_read_timeout interaction with multi-minute mid-stream stalls.
• Moving the control-plane→CAS connection and direct clients to the shared hostname before legacy_grpc_listener: false.

Refs #3628. Spec supersessions: D-017/D-018/D-019 → D-020, R-009 → R-010.

AI assistance: Codex and Claude Code.