Testing Standards
Relevant source files
The following files were used as context for generating this wiki page:
The testing standards within this project establish a multi-layered, automated framework designed to ensure code reliability, performance, and authenticity. The scope ranges from unit testing with Vitest to end-to-end regression suites in Playwright, governed by strict coverage requirements and anti-hallucination gates.
The primary purpose of these standards is to maintain high code quality by enforcing >85% test coverage and utilizing automated tools to detect non-deterministic or "hallucinating" tests that pass falsely during backend failures.
Sources: internal/agents/test-agent-context.md:7-14, README.md:105-115
1. Test Architecture and Layers
The project adopts a 5-layer regression testing model to organize test suites based on dependency depth and business impact.
| Layer | Focus | Components Covered |
|---|---|---|
| Layer 0: Foundation | Infrastructure | Auth, routing, health checks |
| Layer 1: Core | Logic/Persistence | Database, API, business logic |
| Layer 2: Data | Information Flow | Ingestion, validation, transformations |
| Layer 3: UI/UX | Presentation | Components, accessibility, responsiveness |
| Layer 4: Business | Workflows | End-to-end business workflows |
Sources: README.md:105-115, scripts/test-determinism-gate.js:125-132
1.1 Unit and Integration Testing
The project has migrated from Jest to Vitest for unit and integration testing. This decision was driven by Vitest's native ESM support, faster execution via Vite integration, and built-in coverage reporting using the V8 provider. Unit tests focus on pure functions without I/O, while integration tests cover API endpoints and database interactions.
Sources: audits/adhub/decisions/decisions/ADR-001-migrate-to-vitest.md:28-60, internal/agents/test-agent-context.md:46-49
1.2 Performance and Benchmarking
Performance-critical code must include benchmarks. The target for critical paths is often sub-100ms response times, such as the pattern matching target of <50ms for 10k patterns.
Sources: internal/agents/test-agent-context.md:235-243, internal/agents/infrastructure-agent-context.md:88-93
2. Test Execution Workflow
The Test Agent follows a structured "Red-Green-Refactor" cycle to implement and verify tests autonomously.
The diagram above shows the autonomous agent workflow for test implementation, emphasizing the TDD cycle and coverage gates.
Sources: internal/agents/test-agent-context.md:23-42, internal/agents/test-agent-context.md:225-231
3. Anti-Hallucination and Determinism
A "Determinism Gate" is utilized to identify Playwright tests that falsely pass when the backend is unavailable. This gate uses static analysis to scan for anti-patterns and a runtime mode that blocks API routes to verify test authenticity.
3.1 Known Anti-Patterns
The system identifies several "hallucination" triggers that compromise test reliability:
| Pattern ID | Name | Severity | Description |
|---|---|---|---|
| CATCH_SWALLOW | Error Swallowing | CRITICAL | Returns falsy in .catch(), causing silent passes on failures. |
| MULTI_STATUS | Multi-status Acceptance | HIGH | Accepts multiple HTTP codes (e.g., 200 or 404) as success. |
| IF_VISIBLE_GUARD | If-visible Guard | HIGH | Guards assertions with isVisible(), skipping them if elements are missing. |
| FIXTURE_ONLY | Fixture-only Validation | MEDIUM | Validates metadata only, ignoring actual app behavior. |
The sequence diagram illustrates the dual-phase validation of the Determinism Gate, combining code scanning with runtime route blocking.
Sources: scripts/test-determinism-gate.js:14-25, scripts/test-determinism-gate.js:255-270
4. Coverage and Quality Requirements
Strict metrics are enforced via CI/CD pipelines to ensure the integrity of the codebase.
- Coverage Thresholds: Minimum 85% line coverage is mandatory; 90% is the target. 100% of public functions must be tested.
- Determinism: 0% flakiness allowed; tests must be deterministic (e.g., using seeds for random data).
- Infrastructure Performance: Extension activation must occur within <200ms, and service orchestration must remain <500ms.
Sources: internal/agents/test-agent-context.md:82-95, internal/agents/infrastructure-agent-context.md:88-93, audits/legalai-system/phase1-audit.json:96-102
5. Summary
The testing standards prioritize automated enforcement over manual checklists. By migrating to Vitest for speed and ESM compliance, implementing a 5-layer regression model, and utilizing a Determinism Gate to eliminate false positives, the platform ensures that its high testing volume (over 1,500 test cases) provides genuine confidence in system stability.
Sources: audits/adhub/decisions/decisions/ADR-001-migrate-to-vitest.md:154-162, audits/legalai-system/phase1-audit.json:28-32