Testing Strategies
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The shunt project employs a multi-layered testing strategy designed to ensure the reliability of its Claude Code LLM gateway. The strategy focuses on protocol fidelity, streaming preservation, and configuration validation. Because the gateway acts as a proxy between Anthropic's Messages API and various other provider protocols (OpenAI, Codex, Cursor, etc.), testing must guarantee that translation logic does not break streaming semantics or error shapes.
The testing architecture spans from local unit tests within modules to complex integration tests that utilize mock servers to simulate upstream provider behavior. Continuous Integration (CI) enforces strict quality gates, including linting and format checks, to maintain code health across the workspace.
Sources: ARCHITECTURE.md:144-154, AGENTS.md:10-15
Test Categories and Organization
Shunt categorizes tests based on their scope and placement within the repository.
Unit Testing
Unit tests are located within the source files they test, typically inside #[cfg(test)] modules. These tests focus on specific helper functions, data structures, and internal logic that does not cross module boundaries. Key areas for unit testing include routing.rs for model-id resolution and error.rs for Anthropic-compliant error envelope generation.
Sources: ARCHITECTURE.md:144-148, tests/AGENTS.md:14-15
Integration Testing
Integration tests are housed in the tests/ directory and are used when behavior crosses module boundaries or requires external interaction simulation. These tests often use Wiremock to simulate upstream providers and a real Axum server instance to verify gateway HTTP behavior.
Sources: ARCHITECTURE.md:144-148, tests/AGENTS.md:8-12
| Test File | Focus Area | Description |
|---|---|---|
tests/passthrough.rs | Gateway HTTP Behavior | Verifies that unmapped traffic passes through to Anthropic unchanged. |
tests/responses_translate.rs | Protocol Translation | Verifies Anthropic Messages ↔ OpenAI Responses request and SSE conversion. |
tests/ (Other) | Security & Admin | Covers inbound auth, multi-account pooling, and the admin surface. |
Sources: ARCHITECTURE.md:144-148, tests/AGENTS.md:8-12
Testing Workflow and Tooling
The project utilizes standard Rust tooling supplemented by specific mock libraries to ensure environment-independent testing.
The diagram shows the standard developer workflow where code formatting and linting precede unit and integration testing.
Mocking with Wiremock
To avoid live network calls and ensure deterministic results, shunt uses Wiremock to simulate provider protocols. This is critical for testing:
- Request translation and header presence.
- Streaming and non-streaming response handling.
- Tool call and reasoning block support.
- Upstream error and transport failure handling.
Sources: tests/AGENTS.md:17-18, blueprints/provider/_generic.md:65-73
Quality Gates and CI
CI enforces a "warnings-as-errors" policy. Every PR must pass the following commands before it is considered ready:
- Format Check:
cargo fmt --all --check - Lints:
cargo clippy --all-targets --all-features -- -D warnings - Test Suite:
cargo test --all-features --workspace
Sources: AGENTS.md:10-13, CONTRIBUTING.md:7-12
Performance and Caching
Developers may optionally use sccache to speed up local builds. CI utilizes sccache with a GitHub Actions backend to minimize compilation times. Incremental compilation is often disabled in these environments to maximize cache hit rates.
Sources: CONTRIBUTING.md:18-35
Invariants and Regression Testing
The testing strategy is anchored by several architectural invariants that must never be weakened:
- Streaming Preservation: Tests must strictly verify that upstream SSE responses are not buffered unless explicitly requested.
- Error Shaping: Gateway-owned errors must always match the Anthropic error shape (except for specific Codex endpoints).
- Unmapped Traffic: Tests must ensure that requests matching no specific route resolve correctly to the
default_provider(Anthropic).
The sequence diagram illustrates the requirement for unbuffered streaming, which is a primary focus of integration tests.
Sources: ARCHITECTURE.md:104-122, src/AGENTS.md:31-35, tests/AGENTS.md:23-25
New Provider Validation
When implementing a new provider, a specific blueprint requires adding focused protocol integration tests that cover:
- Authentication injection and rejection of unsafe origins.
- Model routing and retry/failover classification.
- Unsupported features returning explicit, correctly shaped errors.
Sources: blueprints/provider/_generic.md:65-74
Summary
The testing strategy for shunt is comprehensive, combining strict CI quality gates with specialized integration tests that use Wiremock to simulate various LLM provider protocols. By focusing on architectural invariants such as unbuffered streaming and Anthropic-compliant error envelopes, the project ensures that the gateway remains a transparent and reliable proxy for Claude Code.
Sources: ARCHITECTURE.md:144-154, tests/AGENTS.md:21-25