fix: long partitions behind a NAT idle timeout never return

Merged
CY
2m ago
+396-29

What

Partition requests that take longer than a few minutes now return their response when the client is behind a NAT or load balancer that drops idle connections. The SDK's own HTTP clients now send TCP keepalive probes while they wait. That covers the default sync and async clients and the split-PDF client.

hi_res partition, response headers after ~5 min, client behind a NAT gateway with a 4 min idle timeout

Before: request sent → no packets for 4 min → NAT drops the flow → response never arrives → ReadTimeout at the full read timeout (900 s)
After:  request sent → keepalive probe every 30 s after 60 s idle → flow stays open → 200

It is on by default (60 s idle, 30 s interval, 5 probes). It can be tuned or turned off with UNSTRUCTURED_CLIENT_TCP_KEEPALIVE, UNSTRUCTURED_CLIENT_TCP_KEEPALIVE_IDLE, UNSTRUCTURED_CLIENT_TCP_KEEPALIVE_INTERVAL and UNSTRUCTURED_CLIENT_TCP_KEEPALIVE_COUNT. The split-PDF plan_created and batch_async_start logs show the setting in use.

Why

Long-running partition requests can exceed a NAT gateway’s idle timeout, causing responses to be lost while the client waits for its read timeout. This change enables TCP keepalive to keep those connections active.

Neither httpx nor the SDK turns on keepalive, and the OS keepalive settings only apply to sockets that ask for it. That makes the SDK, which opens these sockets, the place to fix it.

Unchanged:

  • Clients passed in by the caller (client= / async_client=) are not modified.
  • Proxy routing from HTTP(S)_PROXY / NO_PROXY works as before. Connections made through a proxy do not get keepalive, because httpcore drops socket options for proxied connections.
  • If a future httpx release moves the helper used to rebuild proxy routing, the SDK falls back to building its clients exactly as it did before this change.

Validation

  • Unit tests (416 passed, on top of 0.46.4), pylint and mypy pass.
  • Tests check that real sockets carry SO_KEEPALIVE and every idle, interval and count setting. They also cover the bounds on the environment values, proxy routing and parity of TLS, limits and timeouts.
  • Docker reproduction with a NAT router that silently drops idle flows after 20 s, and a server that answers after 45 s:
    • with keepalive: 200 at 45 s;
    • without keepalive: ReadTimeout at 90 s.
  • Released as 0.46.5: CHANGELOG entry and _version.py bump (0.46.4 is already published).