HTTP Compression

Relevant source files

The following files were used as context for generating this wiki page:

• internal/server/compression.go
• internal/server/compression_middleware.go
• internal/server/service.go
• internal/server/compression_middleware_test.go
• internal/server/compression_test.go
• internal/server/compression_service_test.go

HTTP Compression in kamal-proxy is a per-service feature that enables the proxy to encode responses using gzip, brotli, or zstd without the backend application's involvement. It is designed to reduce payload sizes for compressible media types while maintaining efficiency through fixed compression levels and encoder pooling.

The system is implemented as a middleware that wraps the service request handler. It intelligently decides whether to compress based on client headers, response content type, and a configurable minimum length threshold. Responses that are already encoded, opaque media types, or event streams are passed through untouched to avoid unnecessary CPU overhead or latency issues.

Sources: internal/server/compression.go:10-15, internal/server/service.go:556-560

Architecture and Components

The compression system consists of configuration structures, a negotiation engine, and a specialized http.ResponseWriter wrapper that manages the lifecycle of pooled encoders.

Core Configuration

Compression behavior is defined per-service via the CompressionOptions struct.

Field	Type	Description	Default
Encodings	[]string	Content encodings offered (preferred first).	Empty (Disabled)
MinLength	int64	Minimum response size in bytes to trigger compression.	1024 bytes
ContentTypes	[]string	List of media types to compress (supports wildcards like text/*).	Built-in list

Sources: internal/server/compression.go:49-62, internal/server/compression.go:21-23

The Compression Middleware

The CompressionMiddleware sits outside the service's error pages but inside the ACME handler. This positioning ensures that proxy-generated error pages are compressed, while ACME challenge responses remain untouched.

The middleware initializes a compressingResponseWriter which intercepts the response stream to make a compression decision.

Sources: internal/server/compression_middleware.go:66-70, internal/server/service.go:547-560

Data Flow and Logic

Content Negotiation

The proxy evaluates the Accept-Encoding header against the service's configured encodings. The highest q-value sent by the client wins; in case of ties, the order defined in the service configuration determines preference.

Compression Decision Process

A decision to compress is deferred until the proxy has enough information. The compressingResponseWriter holds back headers until it determines if compression is appropriate based on the following criteria:

1. Status Code: Must allow a body (e.g., not 204 No Content or 304 Not Modified).
2. Content-Range: Partial content responses are never compressed.
3. Content-Encoding: If the target already encoded the response, the proxy does not re-encode it.
4. Content-Type: Must match the compressible allow-list.
5. MinLength: The body must meet the minimum size threshold (default 1024 bytes).

Sources: internal/server/compression_middleware.go:213-228, internal/server/compression_middleware.go:297-302

Encoder Pooling

To manage resource consumption, the proxy maintains a global map of sync.Pool for each supported encoding.

Algorithm	Level	Configuration
Gzip	5	Uses klauspost/compress/gzip
Brotli	4	Uses andybalholm/brotli
Zstd	SpeedDefault	Window size: 1MB; Concurrency: 1

Sources: internal/server/compression_middleware.go:22-30, internal/server/compression_middleware.go:39-58

Implementation Details

Response Modification

When compression is activated, the proxy modifies several HTTP headers to ensure protocol correctness:

• Vary: Adds Accept-Encoding so caches distinguish between encoded and identity representations.
• Content-Encoding: Set to the negotiated algorithm.
• Content-Length: Deleted, as the compressed length differs from the original.
• Accept-Ranges: Deleted, as byte ranges are no longer valid for the modified body.
• ETag: Downgraded to a "weak" ETag (prefixed with W/) because the representation has changed.

Sources: internal/server/compression_middleware.go:244-255, internal/server/compression_middleware.go:328-335

Event Stream Handling

text/event-stream media types are excluded from compression by default. Compressing these streams would buffer data to fill the encoder's window, which breaks the real-time nature of Server-Sent Events (SSE). They are only compressed if an operator explicitly adds them to ContentTypes.

Sources: internal/server/compression.go:138-143, internal/server/compression_middleware_test.go:207-224

Summary

HTTP Compression in kamal-proxy provides a high-performance transparent encoding layer. By utilizing fixed compression levels and encoder pools, it balances CPU usage against bandwidth savings. Its conservative decision logic ensures that streaming protocols and already-compressed media types remain efficient and functional.

Sources: internal/server/compression.go:7-15, internal/server/compression_middleware.go:22-26