Testing Strategy
Relevant source files
The following files were used as context for generating this wiki page:
The testing strategy of the freebuff-proxy project ensures the reliability of its OpenAI-compatible reverse proxy functionality through a multi-layered verification approach. It prioritizes efficiency by mandating full test execution only before final submission, avoiding frequent, time-consuming test cycles during active development.
The system utilizes specialized test suites for smoke testing, contract verification, and end-to-end (E2E) performance monitoring. It enforces strict "falsifiable assertions," requiring developers to verify that tests fail when implementation code is intentionally broken.
Test Execution Architecture
Tests run as independent sub-processes to prevent global state pollution, as many suites mock upstream services or modify global fetch configurations. The main entry point is test/run.ts, which executes suites in a fixed order.
The diagram shows the serial execution flow where each suite is isolated in its own process.
Sources: test/run.ts:9-15, test/run.ts:46-62
Suite Classifications
The project categorizes tests into specific directories based on their responsibilities:
| Category | Path | Responsibility |
|---|---|---|
| Smoke | suites/entries/smoke/ | Mocks upstream to verify main links (session reuse, proxy pools, probe). |
| Verify | suites/entries/verify/ | Checks contracts, tool mappings, parameter translation, and DOM structures. |
| E2E | suites/entries/e2e/ | Performs real browser/service assertions, primarily for frontend loading speeds. |
| Gates | scripts/gates/ | Enforces code quality red lines, documentation audits, and fingerprinting. |
Sources: test/AGENTS.md:23-31, scripts/AGENTS.md:14-22
Core Verification Principles
Falsifiable Assertions
Every significant assertion must be provably correct. Developers must temporarily break the implementation to confirm the test turns "red" before restoring it. This ensures the test is actually measuring the intended behavior.
Sources: CLAUDE.md:73-75, test/AGENTS.md:35
Upstream Contract Alignment
The system maintains a "source of truth" for upstream endpoints and headers in src/upstream/upstream-contract.ts. Tests ensure that all required upstream endpoints found in captures are registered and that no retired headers are reintroduced.
This sequence ensures synchronization between actual network captures and internal code constants.
Sources: scripts/check-upstream-contract.ts:10-23, scripts/check-upstream-contract.ts:79-106
Tooling and Gatekeeping (Ratchet Mechanism)
The project employs a "Ratchet" mechanism for code style and quality. Baselines are stored in .gates/*.json. The gates allow metrics (like full-width punctuation counts) to decrease but never increase.
Code Quality Red Lines
The node scripts/gates/run.ts command executes 17 distinct gates. These include:
- Volume Limits: Backend files $\le$ 300 lines; frontend $\le$ 500 lines.
- Structure: Functions $\le$ 80 lines; mandatory JSDoc for exports.
- Documentation: All registered routes must be documented in
docs/design/api.md.
Sources: CLAUDE.md:65-72, scripts/gates/checks/doc/check-docs.ts:88-100
Frontend and Performance Testing
E2E tests in suites/entries/e2e/ specifically guard the loading speed of the dashboard. They enforce thresholds for the main screen, settings page, and system prompt page. A critical assertion prevents the premature loading of heavy libraries like Monaco (3.7MB) until explicitly required.
The flow ensures that heavy assets do not degrade the initial user experience.
Sources: test/AGENTS.md:46-52, dashboard/AGENTS.md:33-35
Tool Mapping and Parameter Translation
A significant portion of the verification logic is dedicated to translating downstream client signals into official upstream formats. The dispatch-ready.ts suite verifies the entire chain:
- Name Mapping: Downstream tool names are correctly mapped to official names.
- Parameter Translation: Arguments are reshaped to match official schemas (e.g., converting
timeoutMstotimeout_seconds). - Requirement Check: Ensures all mandatory fields for the downstream tool are present after translation.
Sources: src/upstream/signals/AGENTS.md:10-18, test/suites/entries/verify/tool/param/dispatch-ready.ts:10-23
Summary
The testing strategy ensures technical accuracy by anchoring all verification in real network captures and strict code gates. By utilizing a serial execution of isolated suites and a non-negotiable ratchet for code quality, the project maintains a lightweight yet robust architecture. Significance is placed on preventing regression in upstream contracts and protecting frontend performance through lazy-loading assertions.