Custom Key Extraction
Relevant source files
The following files were used as context for generating this wiki page:
Custom Key Extraction is a core feature of the FastLimit library that allows developers to define exactly how a request is identified for rate limiting purposes. While the system defaults to IP-based identification, it provides flexible mechanisms to extract identifiers from any part of an incoming request, such as headers, path parameters, or user session data.
This system ensures that rate limits can be applied granularly, supporting scenarios like per-user limiting, API key throttling, or multi-tenant isolation. The extraction process is designed to be secure, employing URL encoding and hashing to prevent key collisions and ensure Redis compatibility.
Sources: fastlimit/limiter.py:277-313, README.md:195-207
Extraction Logic and Flow
The extraction process begins within the limiter.limit() decorator or the manual check() method. If a custom key function is provided, the library executes it against the request object. If no function is provided or if the custom function fails, the system falls back to a default extraction logic.
Extraction Sequence
The following diagram illustrates how the library determines which identifier to use for a given request.
The sequence prioritizes explicit developer definitions before falling back to network-level identification.
Sources: fastlimit/decorators.py:165-184, fastlimit/decorators.py:207-248
Default Key Identification
When no custom extraction logic is defined, FastLimit relies on the _get_default_key function. By default, it uses the direct IP address of the client. It includes specific logic to handle proxy headers, but only when explicitly configured to do so for security reasons.
| Feature | Logic | Source |
|---|---|---|
| Primary Identifier | request.client.host | Client IP address |
| Proxy (XFF) | request.headers.get("X-Forwarded-For") | Used if trust_proxy_headers is True |
| Proxy (Real IP) | request.headers.get("X-Real-IP") | Used as second proxy fallback |
| Final Fallback | "ip:unknown" | Used if no identifier can be found |
Sources: fastlimit/decorators.py:207-248, tests/test_security.py:108-140
Proxy Security
The library enforces a strict security policy regarding proxy headers. X-Forwarded-For and X-Real-IP are ignored by default to prevent attackers from spoofing their IP to bypass rate limits. They are only utilized if trust_proxy_headers is set to True in the RateLimiter config or the decorator.
Sources: fastlimit/decorators.py:214-222, tests/test_security.py:142-155
Implementation of Custom Extractors
Developers can provide a Callable to the key parameter of the @limiter.limit decorator. This callable receives the request object and must return a string.
Common Extraction Patterns
| Identifier Type | Implementation Example | Source File |
|---|---|---|
| User ID | key=lambda req: f"user:{req.path_params.get('user_id')}" | examples/fastapi_app.py:136 |
| API Key | key=lambda req: req.headers.get("X-API-Key") | examples/multi_tenant.py:151 |
| Tenant ID | key=lambda req: req.headers.get("X-Tenant-ID", "default") | examples/fastapi_app.py:158 |
Security and Sanitization
Once a key is extracted, it is passed through the generate_key utility. This utility performs several critical transformations:
- URL Encoding: Components are URL-encoded to prevent delimiter collision (e.g., a user ID containing a colon
:is encoded so it doesn't break the Redis key structure). - Hashing: If the resulting key is excessively long (default > 200 characters), it is hashed to a deterministic, shorter string to optimize Redis performance.
- Prefixing: The global
key_prefixis prepended to ensure logical isolation in Redis.
Multi-Tenant Key Context
Key extraction can be further namespaced using the tenant_type parameter. While the key identifies the specific entity (e.g., a specific user), the tenant_type identifies the group or tier they belong to (e.g., "premium"). This allows the same identifier to have different limits across different contexts.
- Key Function: Identifies the "who" (e.g., API Key).
- Tenant Function: Identifies the "context" (e.g., Subscription Tier).
Sources: fastlimit/limiter.py:159-178, examples/multi_tenant.py:151-155
Conclusion
Custom Key Extraction in FastLimit provides the necessary flexibility to move beyond simple IP-based rate limiting. By allowing developers to inject custom logic for identifier extraction and combining it with automated sanitization and collision prevention, the system supports complex, production-grade API architectures including multi-tenant SaaS platforms and granular user management.