feat(dwctl): accept gzip, zstd and br request bodies and compressed batch files (DW-985)

Merged
CR
1d ago
+1146-21

Closes part of DW-985 (compression; the remaining 415s follow in a stacked PR).

Why

Clients can't send us compressed bodies today, and the ones that try get a misleading error. A gzipped /ai/v1/chat/completions body reached the model lookup as raw bytes and came back as a 400 saying the model parameter was missing. A .jsonl.gz batch file failed as "invalid UTF-8 on line 1".

Batch files gain the most. Every line repeats the same system prompt and schema, so a batch file compresses about twice as well with zstd or brotli as with gzip (roughly 23–29× against gzip's 12× on a synthetic file). A single chat request compresses about 3–4× whichever codec is used.

What changes

Realtime inference bodies (/ai/v1/*). limit_inference_body now decodes Content-Encoding: gzip (and x-gzip), zstd and br before any other inference layer reads the body. Inference routing, request logging, translation, the cache and onwards all keep seeing plain JSON, and upstreams receive uncompressed bodies: Content-Encoding is removed and Content-Length is set to the decoded size.

  • Size limit: limits.requests.max_body_size applies to the decoded size, since that is what every inner layer buffers. Decoding stops as soon as the output passes the limit, so a small zip bomb costs at most the limit, and it gets a 413.
  • zstd window: zstd allocates the window a frame declares up front, so without a cap a tiny body could claim gigabytes. The realtime decoder caps it at 8 MiB, which RFC 9659 already forbids zstd content-coding encoders from exceeding. A frame that declares a decoded size over the limit gets a 413 straight away.
  • Unsupported codings: any other coding (deflate, lz4, compress, layered codings such as gzip, br) gets a 415. The response includes Accept-Encoding: gzip, zstd, br (RFC 9110 §15.5.16) and code unsupported_content_encoding.
  • Corrupt data: a body that isn't valid data for its declared coding gets a 400 with code body_decode_failed.
  • Mislabelled plain JSON keeps working. Until now the header was ignored, so a plain JSON body under a wrong or meaningless coding worked, e.g. a proxy's stray gzip or a client's utf-8. Such a body is now passed on as plain and counted in dwctl_inference_bodies_mislabelled_total{declared}, instead of getting the 415 or 400 above. It only takes this path when the body fails to decode, so a real compressed body never does.
  • Bounded decoding: decoding runs before authentication, and a few KB of compressed body can expand to the full limit plus a decoder window. So decodes are limited to one per CPU. Requests over the bound wait for a slot rather than failing, so legitimate traffic is at most delayed, and uncompressed bodies never take a slot.
  • Error format: these errors use the same per-API shape as the existing 413: Anthropic's for /messages, OpenAI's otherwise.
  • Metrics: refusals are recorded in onwards_rejections_total like the other layers' refusals. Successful decodes are counted in dwctl_inference_bodies_decoded_total{encoding}.
  • Blocking work: decoding runs on a blocking thread, since inflating a full-size body takes tens of milliseconds.

Batch file uploads (POST /ai/v1/files). A file part that starts with a gzip or zstd magic number is decoded as it streams in, with async-compression's tokio decoders.

  • max_file_size, the JSONL parser and the stored file all see the decoded content. The reported bytes is the decoded size, and batch.jsonl.gz is stored as batch.jsonl.
  • A truncated or corrupt file gets a 400 that names the format, not a UTF-8 error.
  • zstd window for files: capped at 128 MiB, zstd's own default, so any file the zstd CLI decompresses without extra flags also uploads, including --long files. That window is allocated per upload, but uploads are capped by max_concurrent_uploads. Tighten it if that's too much memory for your deployment.
  • xz, bzip2, zip and lz4 files are recognised and refused with a message naming the format. That's a 400 here; the stacked PR changes it to a 415.

Formats are detected from the content, not from the file name or the part's content type, which are often missing or wrong. No JSONL file can start with any of these signatures.

Not in this PR

  • Further 415s, for non-multipart uploads and content types on JSON endpoints: in the stacked PR.
  • Client side, i.e. compressing by default in autobatcher and documenting a compressing transport for the SDKs: to follow.

Testing

  • inference::body_limit: each coding round-trips, at the limit and with limits disabled. Unknown and layered codings get a 415. Plain JSON under any wrong coding passes through (passes_plain_json_under_a_wrong_content_encoding_through, and end to end in both modes). A decode waits while every slot is taken (decodes_wait_for_a_free_slot). Zip bombs get a 413 for all three codecs. Oversized zstd windows are refused. Corrupt data gets a 400. /messages gets the Anthropic shape. Refusals are counted.
  • upload_compression: gzip (including multi-member) and zstd round-trip whatever the chunking. Truncated and corrupt files report a decode error, and upload errors pass through untouched, so length-limit detection still works. Covers format sniffing and filename stripping.
  • files: compressed uploads end to end (stored name, decoded size, download content), and broken and unsupported compression name the problem.