Testing Strategy & Golden Tests
Relevant source files
The following files were used as context for generating this wiki page:
- crates/mcb-server/tests/integration/golden_acceptance_integration.rs
- crates/mcb-server/tests/e2e/golden_tools.rs
- crates/mcb-validate/tests/integration/integration_tests.rs
- crates/mcb-server/tests/golden/mod.rs
- AGENTS.md
- tests/e2e/browse-ui.spec.ts
- crates/mcb-validate/tests/integration/integration_full.rs
Introduction
The testing strategy for the Memory Context Browser (MCB) project is a multi-layered approach designed to ensure architectural integrity, functional correctness, and performance stability. It spans from low-level unit tests to high-level End-to-End (E2E) "Golden Tests" that validate the entire system against real-world fixtures. The strategy prioritizes the use of real local providers, such as FastEmbedProvider and EdgeVec, to simulate production environments without external dependencies.
Golden Tests serve as the project's primary acceptance criteria, validating that repository indexing, semantic search, and Model Context Protocol (MCP) schemas function correctly. These tests are integrated into the development lifecycle via a dedicated Makefile and are enforced through CI gates to prevent regressions in core features like Clean Architecture compliance and search relevance.
Sources: crates/mcb-server/tests/integration/golden_acceptance_integration.rs:1-15, crates/mcb-server/tests/golden/mod.rs:1-25, AGENTS.md:120-135
Testing Hierarchy
The project categorizes tests into several distinct scopes, each targeting a specific layer of the Clean Architecture.
Test Scopes and Verbs
Testing is executed through the canonical make test command, which supports various SCOPE parameters:
| Scope | Description | Target |
|---|---|---|
unit | Low-level logic validation | Individual modules/functions |
golden | Acceptance tests using real fixtures | E2E Tooling & Workflows |
integration | Cross-crate boundary validation | Component interaction |
e2e | Real binary and UI testing | Playwright & Full Binary runs |
validate | Architectural rule enforcement | Dependency graph & SOLID rules |
Sources: AGENTS.md:105-115, crates/mcb-server/tests/golden/mod.rs:1-15
Architecture Validation
MCB utilizes mcb-validate to enforce Clean Architecture rules at compile-time and during the test phase. This includes checking for circular dependencies and forbidden inward-flowing dependencies (e.g., domain depending on infrastructure).
The diagram shows the relationship between the architecture validation rules and the standard testing execution flow.
Sources: crates/mcb-validate/tests/integration/integration_tests.rs:1-30, AGENTS.md:165-180
Golden Acceptance Tests
Golden Tests are the cornerstone of the MCB testing strategy. They validate the "core functionality" using real local providers to ensure that search returns results matching expected files within defined latency limits.
Core Components
Golden tests rely on a structured configuration usually loaded from JSON fixtures (e.g., golden_queries.json).
- TestQuery: Defines the ID, natural language query, expected file results, and performance budgets.
- QueryConfig: Sets global relevance thresholds and
top_klimits for the search engine. - Real Providers: Tests utilize
FastEmbedProviderfor embeddings andEdgeVecfor vector storage to ensure end-to-end integration without network calls.
Sources: crates/mcb-server/tests/integration/golden_acceptance_integration.rs:32-60, crates/mcb-server/tests/e2e/golden_tools.rs:1-10
Workflow Logic
A standard Golden Test follows a complete lifecycle to verify system state transitions.
The sequence diagram illustrates the E2E workflow: clearing the environment, indexing a repository, performing a semantic search, and verifying the output.
Sources: crates/mcb-server/tests/e2e/golden_tools.rs:20-100, crates/mcb-server/tests/integration/golden_acceptance_integration.rs:250-300
User Interface (UI) E2E Testing
For the web-based admin interface, the project uses Playwright to perform browser-based testing. These tests ensure that the "Browse UI" and "Admin Functional" components are responsive and handle real data correctly.
UI Validation Targets
- Keyboard Navigation: Validates
j/kfor chunk navigation andg/Gfor start/end jumps. - Theme Toggle: Ensures persistence of light/dark modes in
localStorage. - Responsiveness: Tests layouts across Mobile (375px), Tablet (768px), and Desktop (1920px) viewports.
- MCP Integration: The UI tests often trigger fresh indexing via MCP POST requests to
localhost:18080before asserting on the rendered HTML content.
Sources: tests/e2e/browse-ui.spec.ts:100-250, tests/e2e/admin-functional.spec.ts:1-50
Quality Metrics and Performance
The testing strategy includes a strict enforcement of code quality metrics using RcaAnalyzer.
Metric Thresholds
MCB enforces limits on code complexity to maintain maintainability:
| Metric | Default Severity | Purpose |
|---|---|---|
| Cognitive Complexity | Warning | Detects deeply nested logic |
| Cyclomatic Complexity | Error | Limits branching paths |
| Function Length | Warning | Encourages modularization |
Sources: crates/mcb-validate/tests/integration/integration_metrics.rs:1-80, crates/mcb-validate/tests/integration/integration_full.rs:300-330
Performance Baselines
Golden tests include max_latency_ms checks for search queries. If a search exceeds the performance budget (e.g., 500ms for local lookups), the test fails, even if the result is semantically correct.
Sources: crates/mcb-server/tests/integration/golden_acceptance_integration.rs:43, crates/mcb-server/tests/integration/golden_acceptance_integration.rs:350-360
Conclusion
The MCB testing strategy ensures that every layer of the system—from the internal domain logic to the external MCP tool surface and Admin UI—is validated through automated, repeatable tests. The reliance on Golden Tests with real local providers guarantees that the software remains "Ready for Agents" by providing a stable and predictable semantic context browser.
Sources: AGENTS.md:120-140, crates/mcb-server/tests/golden/mod.rs:1-25