End-to-End Testing
Relevant source files
The following files were used as context for generating this wiki page:
End-to-end (E2E) testing in the infrahub-ops-cli project validates the functionality of the backup, task manager, and collection tools within real-world deployment environments. These tests verify that the Go-based binaries interact correctly with Infrahub components deployed via Docker Compose or Kubernetes (vcluster). The suite ensures that complex operations, such as troubleshooting-bundle collection and database backups, perform as expected without causing side effects like service restarts or scale events.
Sources: AGENTS.md:12-25, specs/archive/003-collect-tool/tasks.md:13-16
Test Infrastructure and Environment
The project utilizes pytest to orchestrate E2E scenarios. It leverages specialized fixtures to provision Infrahub instances in isolated environments, ensuring tests do not interfere with production data.
Environment Backends
Tests are categorized based on the underlying infrastructure they target:
- Docker Compose: Uses
infrahub-testcontainersto deploy a full Infrahub stack, including Neo4j (database), PostgreSQL (task-manager-db), and application containers. - Kubernetes: Utilizes
vclusterand the official Infrahub Helm chart to simulate a cloud-native deployment.
Key Fixtures
| Fixture | Scope | Description |
|---|---|---|
collect_binary | Session | Builds the infrahub-collect binary using make build before running tests. |
infrahub_container | Function | Provides a running Infrahub Docker container for project-based tests. |
vcluster | Session | Provisions a virtual Kubernetes cluster for k8s marked tests. |
helm_install | Function | Installs the Infrahub Helm chart into the target Kubernetes namespace. |
Sources: tests/e2e/conftest.py:25-80, specs/archive/003-collect-tool/tasks.md:88-91, AGENTS.md:154-159
Troubleshooting Bundle Validation
The collection tool (infrahub-collect) is a primary focus for E2E validation. Tests verify that the tool produces a valid .tar.gz archive containing logs, metrics, and diagnostics.
The diagram shows the lifecycle of a collection E2E test, from binary compilation to final artifact validation.
Sources: tests/e2e/test_docker_collect.py:12-45, specs/archive/003-collect-tool/tasks.md:92-98
Kubernetes-Specific Verification
On Kubernetes, E2E tests validate advanced collection features:
- Multi-replica Logs: Ensures logs are collected for all pod containers when services are scaled (e.g.,
task-workerscaled to 2). - Previous Logs: Verifies that
.previous.logfiles are captured for containers that have experienced restarts. - ReadOnly Guard: Confirms that no pod restarts or scale events occur during the collection process (Principle II).
Sources: tests/e2e/test_k8s_collect.py:15-50, specs/archive/003-collect-tool/tasks.md:99-105, specs/archive/003-collect-tool/opsmill-implement-report.md:75-80
Docker-Specific Verification
Docker tests focus on project discovery and secret handling:
- Project Selection: Validates that the
--projectflag correctly targets specific Docker Compose deployments. - Secret Masking: Inspects environment and configuration dumps to ensure keys like
password,secret,token, orkeyare replaced with********. - Degraded Mode: Verifies the tool exits with code 0 even if a service (like
cache) is stopped, recording the failure in the manifest.
Sources: tests/e2e/test_docker_collect.py:50-85, specs/archive/003-collect-tool/tasks.md:108-115
Bundle Analysis Helpers
To simplify assertions within tests, the project uses helper classes to inspect the generated troubleshooting bundles. The SupportBundle class provides a structured way to access archive contents.
The SupportBundle Class
The SupportBundle helper (found in tests/helpers/bundle.py) implements the following logic:
- Extracts the
bundle_information.jsonmanifest to read the environment and collector results. - Provides methods to read log files for specific services and replicas.
- Calculates checksums or sizes for verification.
This sequence illustrates how E2E tests interact with the SupportBundle helper to validate the internal structure of a collection archive.
Sources: tests/helpers/bundle.py:10-60, tests/e2e/test_docker_collect.py:30-40
Continuous Integration Wiring
E2E tests are integrated into the GitHub Actions workflow (.github/workflows/ci.yml). They are executed using specific pytest markers to target the appropriate runner environment.
| Marker | Requirement | Scenario |
|---|---|---|
docker | Docker Compose | Validates standalone and containerized deployments. |
k8s | vcluster / kubectl | Validates Helm-based deployments. |
enterprise | Enterprise context | Tests CI matrix legs specific to non-Community editions. |
Sources: tests/e2e/conftest.py:10-20, specs/archive/003-collect-tool/opsmill-implement-report.md:45-50, specs/archive/003-collect-tool/tasks.md:130-135
E2E testing provides the final validation layer, ensuring that the infrahub-ops-cli tools maintain operational safety and technical accuracy across diverse infrastructure environments. These tests successfully identify environment-specific regressions, such as log streaming failures or incorrect pod labeling, before release.