Introduction to FastLimit
Relevant source files
The following files were used as context for generating this wiki page:
FastLimit is a high-performance, Redis-backed rate limiting library specifically designed for modern Python applications. It features an async-first design, making it highly suitable for integration with frameworks like FastAPI and Starlette. The library ensures atomicity and eliminates race conditions by utilizing Redis Lua scripts for all rate-limiting operations.
The system is architected to support high-throughput environments, capable of handling over 10,000 requests per second with sub-2ms p99 latency. It offers multiple rate-limiting algorithms, automatic HTTP header injection, multi-tenant support, and Prometheus metrics out of the box.
Sources: README.md:1-18, fastlimit/limiter.py:20-35
Core Architecture
The architecture of FastLimit consists of a Python-based interface that communicates with a Redis backend. The logic for incrementing counters and calculating windows is offloaded to Redis via Lua scripts to maintain strict atomicity in distributed environments.
The diagram above illustrates the request flow from the application through the RateLimiter interface to the atomic Lua execution in Redis.
Sources: README.md:280-295, fastlimit/limiter.py:40-60
Key Components
| Component | Description |
|---|---|
RateLimiter | The primary entry point for the library, managing connections and routing checks to specific algorithms. |
RedisBackend | Handles the low-level communication with Redis and the execution of cached Lua scripts. |
RateLimitHeadersMiddleware | A FastAPI/Starlette middleware that automatically injects rate limit status into response headers. |
RateLimitConfig | Pydantic-based configuration model for global settings like Redis URL and default algorithms. |
Sources: fastlimit/limiter.py:38-55, fastlimit/init.py:31-45, README.md:120-135
Rate Limiting Algorithms
FastLimit provides three production-tested algorithms, each catering to different performance and accuracy requirements.
Algorithm Comparison
| Feature | Fixed Window | Token Bucket | Sliding Window |
|---|---|---|---|
| Burst Handling | Boundary bursts possible (2x) | Smooths traffic; allows controlled bursts | Prevents boundary bursts entirely |
| Accuracy | Good | Good | Best |
| Memory Usage | Lowest (~100 bytes) | Medium (~150 bytes) | Medium (~200 bytes) |
| Complexity | Simple | Moderate | Moderate |
Sources: README.md:105-115, benchmarks/performance.py:270-300
Fixed Window (Default)
This algorithm divides time into fixed buckets (e.g., specific minutes). While simple and memory-efficient, it can allow a "boundary burst" where a user consumes their full limit at the end of one window and the start of the next.
Sources: README.md:65-75, fastlimit/limiter.py:167-180
Token Bucket
Tokens are added to a bucket at a fixed rate. Each request consumes a token. This allows for bursty traffic while maintaining a steady long-term rate. It uses millisecond precision to prevent issues with very low rates.
Sources: README.md:80-90, fastlimit/limiter.py:182-202
Sliding Window
This provides the highest accuracy by combining the current window's count with a weighted portion of the previous window. It ensures that the rate limit is strictly enforced across any moving time frame.
Sources: README.md:95-103, fastlimit/limiter.py:204-221
Key Features
Multi-Tenancy
FastLimit supports isolated limits for different users, organizations, or tiers. This is achieved through dynamic key generation and tenant_type parameters, allowing "Free" and "Premium" tiers to exist on the same endpoint with different quotas.
Sources: examples/fastapi_app.py:145-165, fastlimit/limiter.py:140-155
Cost-Based Limiting
Endpoints can assign a "cost" to requests. For example, an expensive ML inference might count as 10 units, while a simple metadata fetch counts as 1.
Sources: README.md:215-225, examples/fastapi_app.py:170-190
Metrics and Observability
When enabled, FastLimit collects Prometheus metrics to monitor the health and performance of the rate-limiting system.
The diagram shows the standard metrics pipeline for observability.
Sources: README.md:173-195, fastlimit/init.py:47-55
Usage Patterns
Decorator Implementation
The most common usage is via the @limiter.limit decorator on FastAPI route handlers.
@app.get("/api/data")
@limiter.limit("100/minute")
async def get_data(request: Request):
return {"data": "..."}
Sources: README.md:143-150, fastlimit/limiter.py:252-300
Manual API
For complex logic where decorators are insufficient, the manual check and check_with_info methods can be used. check_with_info is optimized to return usage data without requiring a second Redis call.
Sources: fastlimit/limiter.py:110-145, fastlimit/limiter.py:380-400
Performance Benchmarks
The library is optimized through integer-only math (using a 1000x multiplier for precision) and caching Lua scripts via EVALSHA.
| Metric | Target |
|---|---|
| p50 Latency | < 0.8ms |
| p99 Latency | < 2.0ms |
| Throughput (Concurrent) | 10,000+ req/s |
Sources: README.md:235-245, benchmarks/performance.py:30-45
Summary
FastLimit provides a robust, atomic, and high-performance solution for rate limiting in Python. By leveraging Redis and Lua, it maintains consistency across distributed deployments while offering the flexibility of multiple algorithms and tiered support for multi-tenant SaaS applications.