Testing Strategy & CI
Relevant source files
The following files were used as context for generating this wiki page:
Testing in Infrahub ensures the stability of a graph-based infrastructure platform through a multi-tiered approach covering backend logic, frontend components, and end-to-end user workflows. The project uses a combination of Python-based testing for the backend and TypeScript-based testing for the frontend, integrated via a Continuous Integration (CI) pipeline that validates all changes before they reach the stable branch.
The strategy prioritizes high-fidelity reproduction of bugs through failing tests and maintains strict linting and formatting standards to ensure code quality across all domains.
Testing Tiers and Tools
Infrahub categorizes tests into specific tiers based on the scope of the code they exercise and the environment required for execution.
Backend Testing
The backend utilizes pytest for various test types. Integration and component tests often require a running Docker daemon as they utilize testcontainers to manage backing services like Neo4j.
| Test Type | Directory | Tool | Description |
|---|---|---|---|
| Unit | backend/tests/unit/ | pytest | Tests isolated functions and domain logic. |
| Component | backend/tests/component/ | pytest | Tests specific components with backing services via testcontainers. |
| Functional | backend/tests/functional/ | pytest | Tests API endpoints and broader application flows. |
| Integration | backend/tests/integration_docker/ | pytest | Validates full system integration within Docker environments. |
Sources: backend/AGENTS.md:17-21, dev/bug-pipeline/test-writing.md:46-48, AGENTS.md:52-54
Frontend Testing
The frontend employs a dual-tool strategy to handle both isolated logic and browser-based interactions.
| Test Type | Tool | Location | Description |
|---|---|---|---|
| Unit | Vitest | Colocated (.test.ts) | Tests TypeScript utility functions and data transformations. |
| Component | Vitest | Colocated (.test.ts) | Tests React components in browser mode. |
| E2E | Playwright | tests/e2e/ | Full user workflow validation against a live stack. |
Sources: frontend/app/AGENTS.md:10-20, dev/bug-pipeline/test-writing.md:49-51
Test Execution Flow
The following diagram illustrates the workflow for executing different test suites and the environment dependencies involved.
The diagram shows how developers select specific environments and tools to run tests, with backend and E2E tests having specific Docker infrastructure requirements.
Sources: AGENTS.md:52-66, dev/guides/frontend/writing-e2e-tests.md:7-12
Bug Reproduction and Test Writing
The project follows a strict protocol for fixing bugs, which requires writing a failing test before implementing a fix.
- Branch Creation: Create a branch named
ai-bug-pipeline-<issue>-<slug>fromorigin/stable. - Assert Expected Behavior: Write a test that asserts the correct behavior, not the buggy one. The test must fail on existing code.
- Use Production Paths: Tests must exercise actual production code paths rather than internal implementation details or reimplementations.
- Verification: Verify the test fails with an assertion error directly related to the root cause.
- Format and Lint: Format Python code with
uv run invoke formatand frontend code withpnpm biome:fix.
Sources: dev/bug-pipeline/test-writing.md:13-20, 53-86, backend/AGENTS.md:82-84
Frontend Unit Testing Standards
Unit tests for frontend logic follow a Behavior-Driven Development (BDD) structure.
GIVEN/WHEN/THEN Pattern
Every test must contain exactly one of each marker to ensure clarity and single-phase validation.
- GIVEN: Set up test data using factories from
tests/fake/. - WHEN: Execute the function being tested.
- THEN: Assert the expected outcome using specific matchers like
.toBe()or.toEqual().
The sequence diagram details the BDD flow for a frontend unit test, emphasizing the use of factories for data setup.
Sources: dev/guides/frontend/writing-unit-tests.md:53-70
End-to-End (E2E) Testing Strategy
E2E tests live in tests/e2e/ at the repository root. They are written in Python using pytest-playwright and follow an asynchronous execution model.
Minimum Bar for New Pages
Every new page under frontend/app/src/pages/ must ship with at least one happy-path E2E test. This test must:
- Navigate to the page.
- Perform a primary user action.
- Assert the resulting rendered output (e.g., rows or nodes).
Data Isolation and Branches
To prevent side effects between tests, every E2E test file creates its own unique branch via a fixture using generate_random_branch_name(). Tests depend on the shallowest data slice required (e.g., data_rbac or data_sites) to maintain performance.
Sources: dev/guides/frontend/writing-e2e-tests.md:21-30, 89-95, 102-108
Continuous Integration (CI) Pipeline
CI validates the repository through several automated gates. Developers are encouraged to run /pre-ci locally to execute these checks before pushing.
- Linting: Runs
ruffandmypyfor Python,biomefor frontend, andmarkdownlintfor documentation. - Generated Files: Validates that schema definitions, protocols, and documentation are up to date via
uv run invoke docs.validate. - Frontend Gates: Includes
biome cifor linting,knipfor unused dependencies, andbetterer cifor TypeScript regression checks. - Test Sharding: E2E tests are partitioned into shards (e.g.,
shard_foundation,shard_sites_a) to allow parallel execution in CI environments.
Sources: AGENTS.md:72-84, 114-118, frontend/app/AGENTS.md:24-30, dev/guides/frontend/writing-e2e-tests.md:81-85
Conclusion
Infrahub's testing strategy centers on reliability and developer productivity. By combining rigorous backend integration tests with BDD-driven frontend units and comprehensive E2E workflows, the project ensures that both core infrastructure logic and user-facing interfaces function correctly. The use of Docker-based test environments and strict CI gates prevents regressions and maintains a high standard for code quality.