End-to-End Testing
Relevant source files
The following files were used as context for generating this wiki page:
End-to-end (e2e) testing in the pmm-dump project is designed to verify the complete data transfer workflow—from exporting metrics and QAN data from a source PMM (Percona Monitoring and Management) server to importing them into a target PMM server. These tests ensure that the tool functions correctly in a real-world environment involving Docker containers, network communication, and database consistency.
The testing suite utilizes Go's testing framework combined with Docker to orchestrate temporary PMM environments. The scope includes validating data integrity, checking compatibility across different PMM versions, and ensuring that various export filters (such as dashboard or instance-specific filters) behave as expected.
Sources: internal/test/README.md:1-5, README.md:109-115
Test Architecture and Orchestration
The e2e testing framework is built around a deployment controller that manages the lifecycle of PMM instances. Each test typically involves deploying one or more PMM servers, executing the pmm-dump binary, and performing assertions on the results.
Execution Flow
When e2e tests are initiated, the framework follows a specific sequence to ensure a clean testing environment:
- Cleanup: Existing e2e test deployments are removed.
- Environment Setup: Required Docker images are pulled.
- Execution: Tests are launched concurrently.
- Teardown: If tests pass, deployments are deleted; if a test fails, deployments are kept to facilitate troubleshooting.
Sources: internal/test/README.md:44-50, internal/test/e2e/main_test.go:34-63
Lifecycle Diagram
The following diagram illustrates the lifecycle of an e2e test run, managed by the TestMain function and the deployment controller.
Sources: internal/test/e2e/main_test.go:44-64
Environment Configuration
Tests require specific prerequisites and can be executed via make or the go test command.
Configuration Methods
| Method | Command | Description |
|---|---|---|
| Preferred | make run-e2e-tests | Automates config creation, binary building, and test execution. |
| Manual (Single) | go test -v -tags e2e -run ^TestName$ | Runs a specific test using the e2e build tag. |
| Manual (All) | go test -v -tags e2e ./... | Runs all tests in the repository marked with the e2e tag. |
Sources: internal/test/README.md:7-40
Environment Variables
Testing behavior is tuned using environment variables:
PMM_DUMP_MAX_PARALLEL_TESTS: Controls concurrency (default: 4).PMM_DUMP_USE_EXISTING_PMM: If set totrue, the framework skips deploying new containers and uses an existing environment.
Sources: internal/test/README.md:20-22, internal/test/deployment/pmm.go:94-96
Deployment Controller
The deployment package provides the logic to spin up PMM servers, PMM clients, and database instances (like MongoDB) within Docker.
Key Components
- Controller: Manages PMM objects and overall test state.
- PMM Object: Represents a PMM deployment, including its URL, ports (HTTP, HTTPS, Clickhouse), and container IDs.
- Network: Each test deployment typically operates within its own Docker network labeled with
com.percona.pmm-dump.test.
Sources: internal/test/deployment/pmm.go:42-65, internal/test/deployment/pmm.go:193-200
PMM Deployment Sequence
Validation and Data Integrity
A critical aspect of e2e testing is validating that data exported from one instance matches the data imported into another.
Validation Logic
The validate_test.go file implements a deep comparison of chunks:
- VictoriaMetrics: Compares metric hashes, timestamps, and values. It calculates a "loss" percentage to ensure data consistency remains within acceptable thresholds (e.g., loss < 0.1%).
- ClickHouse (QAN): Reads TSV data from exported chunks and compares records using SHA256 hashes to identify missing or altered entries.
Sources: internal/test/e2e/validate_test.go:91-144, internal/test/e2e/validate_test.go:146-177
Data Comparison Summary
| Data Type | Validation Method | Failure Criteria |
|---|---|---|
| VictoriaMetrics | Value & Timestamp matching | Loss > 0.001% or > 5 missing chunks |
| ClickHouse (QAN) | Record hashing (SHA256) | Any missing or different record |
Sources: internal/test/e2e/validate_test.go:81-87, internal/test/e2e/validate_test.go:173-177
Specialized Test Cases
The e2e suite includes specialized tests for specific features:
QAN Filtering
Tests verify the --where and --instance flags by exporting QAN data and validating that the resulting chunks only contain records matching the filter criteria (e.g., service_name='mongo').
Sources: internal/test/e2e/qan_test.go:88-142
Content Limits
The TestContentLimit verifies that the --vm-content-limit flag correctly handles Nginx's client_max_body_size restrictions. It simulates a scenario where a large chunk is blocked by the server and ensures the tool can split the chunk into smaller parts for successful import.
Sources: internal/test/e2e/contentlimit_test.go:37-77
Compatibility
TestPMMCompatibility iterates through multiple PMM versions (defined in versions.yaml) to ensure that pmm-dump can successfully export from and import into different versions of the PMM server.
Sources: internal/test/e2e/version_test.go:34-70
Summary
End-to-End testing provides the final layer of confidence for the pmm-dump tool. By automating the deployment of complex PMM environments and performing granular data validation, the project ensures that metrics and Query Analytics data are transferred reliably across different server configurations and versions.