Performance Benchmarks
Relevant source files
The following files were used as context for generating this wiki page:
Performance benchmarks in SeaStreamer are designed as micro-benchmarks to measure the throughput and efficiency of different streaming backends. The primary goal of these benchmarks is to track performance improvements and prevent regressions during the development of the toolkit. These benchmarks evaluate how the system handles high-volume message processing across various storage and transport layers, including Redis, Files, and Standard I/O.
The benchmarking suite focuses on realistic stream processing scenarios, such as producing and consuming large batches of messages (e.g., 100k messages) with specific payload sizes (e.g., 256 bytes) to determine the baseline capabilities of each backend.
Sources: benchmark/README.md:1-5, benchmark/README.md:19-23
Benchmark Architecture and Components
The benchmarking system is organized as a dedicated crate within the SeaStreamer workspace. It leverages the generic SeaStreamer API to interact with multiple backends using a unified interface.
Key Components
| Component | Description |
|---|---|
sea-streamer-benchmark | The primary package containing micro-benchmark binaries. |
baseline | A binary used to establish the minimum overhead of the system. |
producer | Simulates high-speed message generation and transmission to a backend. |
consumer | Measures the speed of message retrieval and processing from a backend. |
relay | Evaluates the performance of intermediary processing or redirection. |
Sources: benchmark/Cargo.toml:1-12, benchmark/README.md:8-16
Execution Flow
The following diagram illustrates the typical data flow during a performance benchmark run using the stdio and relay components to measure pipeline overhead.
The diagram shows how messages flow through a piped sequence of processes to evaluate the performance of the toolkit's buffering and redirection logic.
Sources: benchmark/README.md:16, README.md:310-312
Backend-Specific Performance Characteristics
Each SeaStreamer backend exhibits unique performance profiles based on its underlying implementation.
Redis Backend
The Redis implementation focuses on high concurrency and throughput. It utilizes a mutex-free implementation achieved through message passing. Key performance features include:
- Pipelined XADD: Batches write operations to Redis.
- Paged XREAD: Optimizes message retrieval for higher throughput.
- Throughput: Capable of processing in the realm of 100k messages per second.
Sources: sea-streamer-redis/Cargo.toml:1-12, README.md:218-223
File Backend
SeaStreamer File is designed for efficient seeking and real-time replay. It avoids the overhead of encoding/decoding message delimiters by using a custom framing format and "Beacons" for indexing.
- Beacon Intervals: Placed at fixed intervals (default 1MB) to allow efficient fast-forward and seeking without scanning the entire file.
- Binary Support: Unlike Stdio, the File backend handles raw binary data without UTF-8 validation overhead.
Sources: sea-streamer-file/README.md:13-18, sea-streamer-file/src/lib.rs:77-79
Standard I/O (Stdio) Backend
Stdio is used for real-time stream processing via Unix pipes. While limited to UTF-8 text, it provides a zero-setup method for measuring the baseline overhead of the async runtime and trait abstractions.
Sources: README.md:255-259
Benchmark Configuration and Runtimes
The benchmarks can be executed under different async runtimes to compare their performance impact on stream processing.
Supported Runtimes
- Tokio: Enabled via the
runtime-tokiofeature. - Async-std: Enabled via the
runtime-async-stdfeature.
Dependency Stack
The diagram illustrates the relationship between the benchmark suite, the core toolkit, and the underlying async runtimes.
Sources: benchmark/Cargo.toml:22-29, Cargo.toml:44-53
Summary of Benchmark Metrics
Benchmarks generally target a standardized load to ensure comparability across different environments.
| Metric | Standard Value |
|---|---|
| Message Count | 100,000 messages |
| Payload Size | 256 bytes per message |
| Total Data Volume | ~30 MB (including headers/overhead) |
| Performance Target (Redis) | ~100k messages/sec |
Sources: benchmark/README.md:19-23, README.md:223
The performance benchmarks provide a critical baseline for the SeaStreamer project. By using the time utility against specialized binaries like producer and consumer, developers can validate that architectural changes, such as the introduction of the Beacon system in the File backend or pipelining in the Redis backend, result in measurable performance gains.
Sources: benchmark/README.md:8-16, sea-streamer-file/README.md:15-18