Test Infrastructure Setup

The test infrastructure for pmm-dump is designed to provide a reproducible environment for End-to-End (e2e) and unit testing. It relies heavily on Docker and Docker Compose to orchestrate PMM Server, PMM Client, and database instances (such as MongoDB) to simulate real-world data transfer scenarios.

The primary goal is to validate the export and import capabilities of the pmm-dump tool across different PMM versions and configurations. The infrastructure supports concurrent test execution and provides mechanisms for automated environment cleanup and troubleshooting.

Sources: internal/test/README.md:1-10, README.md:145-155

Environment Initialization

Before running tests, the environment must be initialized to create necessary configuration files. This is handled by a shell script that generates .env files containing Docker image versions and connection strings.

Configuration Files

The initialization process creates two primary environment files to support tests that require multiple PMM instances (e.g., exporting from one and importing to another).

Sources: setup/test/init-test-configs.sh:1-40, internal/test/README.md:18-24

Initialization Logic

The following diagram illustrates the flow of setting up the test environment variables:

The script ensures that PMM_VERSION, PMM_SERVER_IMAGE, and MONGO_IMAGE are defined for the Docker Compose orchestration.
Sources: setup/test/init-test-configs.sh:23-35

Deployment Orchestration

The project uses a Go-based deployment package to programmatically manage Docker containers during test execution. This allows tests to spin up isolated environments, perform operations, and tear them down automatically.

Deployment Components

The infrastructure orchestrates three main service types as defined in the Docker Compose configuration:

• PMM Server: The central metrics storage and management service.
• PMM Client: The agent responsible for collecting metrics and connecting to the server.
• MongoDB: A sample database service used to generate QAN (Query Analytics) data for testing.

Sources: docker-compose.yaml:1-55, internal/test/deployment/pmm.go:190-230

Orchestration Sequence

The deployment.Controller manages the lifecycle of PMM instances. The sequence below shows how a test deployment is established:

E2E Testing Framework

End-to-End tests are executed using the e2e build tag. The framework includes a global TestMain function that manages the high-level lifecycle of the entire test suite.

Global Lifecycle

1. Cleanup: Removes any pre-existing e2e deployments using deployment.DestroyAll.
2. Pre-pull: Pulls all necessary Docker images to avoid timeouts during individual tests.
3. Execution: Runs all tests tagged with //go:build e2e.
4. Final Cleanup: Destroys deployments if tests pass. If a test fails, deployments are kept for troubleshooting.

Sources: internal/test/e2e/main_test.go:33-65, internal/test/README.md:46-52

Parallelism Control

Parallel execution is supported and configured via environment variables to manage resource consumption on the host machine.

Sources: internal/test/README.md:27-35

Service Configuration Details

The services are configured with specific ports and volumes to ensure they can be accessed by the pmm-dump binary during tests.

Port Mapping

The controller dynamically tracks published ports for the following services to facilitate communication:

• PMM HTTP/HTTPS: Accessed via pmm.PMMURL().
• ClickHouse: The QAN data source, accessed via pmm.ClickhouseURL().
• MongoDB: Data generator, accessed via pmm.MongoURL().

Sources: internal/test/deployment/pmm.go:121-164, docker-compose.yaml:17-21

// From internal/test/deployment/pmm.go
func (p *PMM) ClickhouseURL() string {
	u, err := url.Parse(p.PMMURL())
	if err != nil {
		p.t.Fatal(err)
	}
	u.User = url.UserPassword("default", "clickhouse")
	u.Scheme = "clickhouse"
	u.Path = "pmm"
    // ... logic to append clickhouse port
	return u.String()
}

Sources: internal/test/deployment/pmm.go:142-155

Summary

The test infrastructure provides a robust, containerized environment that automates the deployment of PMM stacks. By utilizing docker-compose.yaml for service definitions and a Go-based deployment controller, the project ensures that pmm-dump is tested against real service instances. The ability to retain failed deployments and configure parallel execution makes this setup suitable for both local development and CI/CD pipelines.