Benchmarking and Performance

Relevant source files

The following files were used as context for generating this wiki page:

• benchmarks/performance.py
• README.md
• CONTRIBUTING.md
• tests/test_fixed_window.py
• examples/algorithms_demo.py
• pyproject.toml

Benchmarking and performance analysis are critical components of the FastLimit project, ensuring the library meets the high-throughput and low-latency requirements of modern production environments. The system is designed to provide near-instantaneous rate limit checks by leveraging Redis-backed atomic operations and optimized Python logic.

The primary goal of the benchmarking suite is to validate performance targets, which include p99 latencies under 10ms and concurrent throughput exceeding 10,000 requests per second. These benchmarks evaluate various dimensions such as algorithm-specific overhead, multi-tenant isolation performance, and accuracy under high-concurrency load.

Sources: benchmarks/performance.py:10-18, README.md:144-154

Performance Targets and Metrics

FastLimit defines specific performance targets to ensure its suitability for high-performance services. These targets are used as a baseline for all benchmark tests.

Metric	Target	Description
p50 Latency	< 2ms	The median time taken for a single rate limit check.
p99 Latency	< 10ms	The upper bound latency for 99% of requests.
Sequential Throughput	> 3,000 req/s	Throughput when requests are processed one after another.
Concurrent Throughput	> 10,000 req/s	Throughput when requests are processed in parallel using asyncio.
Memory per Key	< 200 bytes	Approximate Redis memory consumption for a single rate limit key.
Accuracy	100%	Ensuring exactly the configured limit is allowed, no more or less.

Sources: benchmarks/performance.py:12-18, benchmarks/performance.py:38-44

Benchmarking Architecture

The benchmarking system is encapsulated in the PerformanceBenchmark class, which provides a structured environment for setting up Redis connections, executing various test suites, and teardown operations.

Benchmark Workflow

The following diagram illustrates the lifecycle of a benchmark run, from environment setup to final summary reporting.

The benchmark workflow ensures that environment setup and teardown are handled gracefully while systematically iterating through performance dimensions.
Sources: benchmarks/performance.py:48-64, benchmarks/performance.py:330-357

Algorithm Performance Comparison

Performance varies across the three core algorithms (Fixed Window, Token Bucket, and Sliding Window) due to differences in computational complexity and Redis operations.

Observed Latency and Throughput

Benchmarks consistently show that Fixed Window is the most performant, while Sliding Window provides the highest accuracy at a slight cost to throughput.

Algorithm	p50 Latency	p99 Latency	Throughput	Memory (Approx)
Fixed Window	0.8ms	2.0ms	15,000+ req/s	~100 bytes
Token Bucket	1.0ms	2.2ms	12,000+ req/s	~150 bytes
Sliding Window	1.2ms	2.8ms	10,000+ req/s	~200 bytes

Sources: README.md:144-154, benchmarks/performance.py:221-255

Latency Measurement Logic

Latency is measured by sampling individual requests and calculating statistical distributions using the statistics module.

for i in range(samples):
    key = f"latency:{i}"
    start = time.perf_counter()
    await self.limiter.check(key=key, rate=rate)
    latency = (time.perf_counter() - start) * 1000  # Convert to ms
    latencies.append(latency)

Sources: benchmarks/performance.py:114-123

Throughput and Scalability

FastLimit is tested for its ability to scale across concurrent clients and multiple tenants.

Concurrent Client Scaling

The benchmark_concurrent_clients suite evaluates how throughput changes as the number of clients increases from 1 to 1,000. It uses asyncio.gather to simulate simultaneous requests from multiple simulated client IDs.

The sequence diagram demonstrates the flow of concurrent requests during a throughput test.
Sources: benchmarks/performance.py:141-171

Multi-Tenant Performance

The multi-tenant benchmark simulates a realistic SaaS workload with varying tier distributions (70% free, 25% premium, 5% enterprise). This ensures that key isolation and prefixing do not introduce significant overhead as the tenant count increases.
Sources: benchmarks/performance.py:202-219

Optimization Techniques

The project employs several low-level optimizations to maintain high performance as noted in the source documentation and implementation:

• Lua Script Caching: Uses EVALSHA instead of EVAL to reduce the overhead of sending script code to Redis for every check.
• Atomic Operations: All logic for incrementing counters and checking windows is performed within Redis to prevent race conditions without requiring client-side locks.
• Integer Arithmetic: Uses integer-only math (utilizing a multiplier for precision) to avoid the overhead of float conversions in performance-critical paths.
• Connection Pooling: Maintains a pool of up to 50 connections to Redis to minimize the latency of opening new sockets.

Sources: README.md:156-161, CONTRIBUTING.md:460-482, benchmarks/performance.py:11-18

Benchmarking Tools

The project includes specific dependencies for performance analysis and monitoring, configured in the pyproject.toml.

Tool	Purpose
locust	Load testing and user simulation.
matplotlib	Visualization of benchmark results.
prometheus-client	Collection of real-time performance metrics (latencies, operation counts).

Sources: pyproject.toml:48-55, README.md:123-134

Conclusion

FastLimit's benchmarking suite provides a comprehensive framework for validating the library's performance claims. By testing across multiple algorithms, concurrency levels, and tenant scenarios, the project ensures that its Redis-backed architecture consistently delivers sub-2ms latencies and high-throughput reliability required for production-grade API rate limiting.