Rate Limiting and Retries

Relevant source files

The following files were used as context for generating this wiki page:

• infrahub_sdk/rate_limit.py
• infrahub_sdk/retry.py
• infrahub_sdk/exceptions.py
• specs/ihs-249-sdk-429-retry/plan.md
• specs/ihs-249-sdk-429-retry/tasks.md
• specs/ihs-249-sdk-429-retry/opsmill-implement-report.md

The Infrahub SDK implements a transparent retry mechanism to handle HTTP 429 (Too Many Requests) responses from the Infrahub API. This system ensures that transient rate-limit spikes do not disrupt automation scripts by automatically pausing and retrying requests using jittered exponential backoff or server-provided Retry-After headers.

The mechanism is available in both the InfrahubClient (async) and InfrahubClientSync (sync) clients. It covers all major request paths, including standard queries, mutations, multipart uploads, and streaming initiations. If a request exceeds the maximum retry budget, the SDK raises a specialized RateLimitError.

Sources: specs/ihs-249-sdk-429-retry/plan.md:9-14, specs/ihs-249-sdk-429-retry/tasks.md:27-31

Architecture and Components

The rate-limiting architecture separates decision logic from I/O execution. A pure, I/O-free handler determines backoff durations, while client-specific drivers execute the actual delays (sleeping) and retries.

Core Components

Component	Description
RateLimitRetryHandler	Logic class that parses headers, computes backoff, and decides if a retry should occur.
RateLimitError	Specialized exception raised when retry attempts are exhausted.
Retry Driver	Client-side methods (_send_with_rate_limit_retry) that wrap HTTP requests and manage the retry loop.
Config	Pydantic-based configuration containing retry tunables.

Sources: specs/ihs-249-sdk-429-retry/plan.md:16-20, specs/ihs-249-sdk-429-retry/tasks.md:46-60

Logic Flow

The following diagram illustrates the retry logic executed by the SDK when an HTTP 429 response is received.

Implementation Details

RateLimitRetryHandler

The RateLimitRetryHandler in infrahub_sdk/rate_limit.py performs all calculations without performing any I/O. It provides methods to:

• Parse Retry-After: Supports both delta-seconds (integers) and HTTP-date formats.
• Compute Backoff: Uses the formula min(backoff_max, backoff_base * 2**attempt).
• Apply Jitter: Generates a random uniform delay between 0 and the calculated ceiling to prevent thundering herd issues.
• Evaluate Budget: Checks if the current attempt count is within rate_limit_max_retries.

Sources: specs/ihs-249-sdk-429-retry/tasks.md:54-60, infrahub_sdk/rate_limit.py

Client Drivers

Both InfrahubClient and InfrahubClientSync implement a _send_with_rate_limit_retry driver. The async version uses asyncio.sleep, while the sync version uses time.sleep. These drivers wrap three specific send sites per client:

1. _request: Standard JSON queries and mutations.
2. _request_multipart: File uploads. This site specifically rewinds file objects using seek(0) before retrying to ensure the full body is re-sent.
3. _get_streaming: Stream initiation.

Sources: specs/ihs-249-sdk-429-retry/plan.md:47-51, specs/ihs-249-sdk-429-retry/tasks.md:69-79

Configuration

Retry behavior is controlled via four fields added to ConfigBase. These settings are shared between async and sync clients.

Configuration Field	Type	Default	Description
rate_limit_retry_enabled	bool	True	Enables or disables the transparent 429 retry mechanism.
rate_limit_max_retries	int	5	Maximum number of retry attempts before giving up.
rate_limit_backoff_base	float	0.5	Base value in seconds for exponential backoff.
rate_limit_backoff_max	float	60.0	Maximum wait time in seconds for a single retry attempt.

Sources: specs/ihs-249-sdk-429-retry/tasks.md:46-51, specs/ihs-249-sdk-429-retry/plan.md:28-30

Error Handling

When the retry budget is exhausted, the SDK raises a RateLimitError. This exception is defined in infrahub_sdk/exceptions.py.

• Attributes: Stores the url of the request, the number of attempts made, and the retry_after duration if provided by the server.
• Chaining: The original httpx.HTTPStatusError that triggered the final failure is attached as the __cause__ of the RateLimitError.
• Logging: The SDK logs a WARNING for every retry attempt, including the URL, attempt number, and the scheduled delay.

Sources: specs/ihs-249-sdk-429-retry/tasks.md:52-53, specs/ihs-249-sdk-429-retry/plan.md:65-67, infrahub_sdk/exceptions.py

Summary

The rate limiting and retry system provides automated resilience against Infrahub API rate limits. By centralizing decision logic in a pure handler and implementing symmetric drivers for async and sync operations, the SDK ensures consistent behavior. Users can tune the mechanism through configuration or disable it entirely to handle 429 responses manually.