Getting Started
Relevant source files
The following files were used as context for generating this wiki page:
PMM Dump is a specialized utility designed to transfer metrics and Query Analytics (QAN) data between Percona Monitoring and Management (PMM) Server instances. It facilitates troubleshooting by allowing engineers to export data from a production environment and import it into a local or staging PMM instance for analysis.
The tool supports various data sources within PMM, including VictoriaMetrics for core metrics and ClickHouse for QAN data. It handles the complexities of authentication, data chunking, and load management to ensure minimal impact on the source PMM server during the export process.
Sources: README.md:3-5, cmd/pmm-dump/main.go:61-75
Installation and Building
To use PMM Dump, you must build the binary from the source. The project requires Go 1.21 or higher.
Prerequisites
- Go 1.21+ installed
- Docker (for local development and testing)
makeutility
Build Instructions
Run the following command in the root directory of the repository to generate the binary:
make build
This produces a pmm-dump executable that can be run directly from the command line.
Sources: README.md:7-11, internal/test/README.md:13-16
Basic Usage
The operation of PMM Dump is primarily divided into two main commands: export and import.
Exporting Data
The export command retrieves data from a PMM server and saves it into a compressed .tar.gz archive.
./pmm-dump export --pmm-url "http://USER:PASS@HOST"
Importing Data
The import command takes a previously created dump file and uploads the data into a target PMM server.
./pmm-dump import --pmm-url "http://USER:PASS@HOST" --dump-path FILENAME.tar.gz
Sources: README.md:13-21, cmd/pmm-dump/main.go:124-126, cmd/pmm-dump/main.go:240-242
Workflow Architecture
The following diagram illustrates the high-level data flow when using PMM Dump to transfer data between two PMM environments.
This diagram shows how pmm-dump acts as an intermediary, extracting data from database backends and packaging it into a portable format.
Sources: README.md:13-15, cmd/pmm-dump/main.go:210-235, cmd/pmm-dump/main.go:310-330
Configuration and Flags
PMM Dump provides extensive configuration options via command-line flags and environment variables.
General Options
These flags are applicable to both export and import commands.
| Flag | Envar | Description |
|---|---|---|
--pmm-url | PMM_URL | Full URL of the PMM instance with credentials. |
--pmm-user | PMM_USER | PMM credentials username. |
--pmm-pass | PMM_PASS | PMM credentials password. |
--dump-core | - | If true, processes core metrics (VictoriaMetrics). |
--dump-qan | - | If true, processes QAN metrics (ClickHouse). |
--workers | - | Number of concurrent workers for import/export. |
--allow-insecure-certs | - | Allows connections to PMM via self-signed certificates. |
Export Filtering
Filtering allows for targeted data extraction based on timeframes or specific services.
| Flag | Description | Example |
|---|---|---|
--start-ts | Start RFC3339 date-time for the export. | 2023-01-01T00:00:00Z |
--end-ts | End RFC3339 date-time for the export. | 2023-01-01T12:00:00Z |
--instance | Filter by service name or ID. | mysql-production |
--dashboard | Filter by Dashboard name (VM only). | MySQL Instances Overview |
--ts-selector | Direct Timeseries selector for VictoriaMetrics. | {service_name="mongo"} |
Sources: README.md:27-100, cmd/pmm-dump/main.go:48-115
Load Management
During export, pmm-dump can monitor the source PMM server's load to prevent performance degradation. This is managed through "load thresholds."
Threshold Types
- CPU: Overall CPU load of the PMM instance.
- RAM: Memory usage of the PMM instance.
- MYRAM: Memory usage of the machine running
pmm-dump.
Logic Flow
- Max Load: If reached, the tool postpones the export and waits for the load to drop.
- Critical Load: If reached, the tool stops the export process entirely to protect the source.
Advanced Pipelines
PMM Dump supports streaming data through STDOUT, enabling direct transfer between servers without an intermediate file.
./pmm-dump export --pmm-url="http://source-pmm" --stdout | ./pmm-dump import --pmm-url="http://target-pmm"
This is particularly useful when local disk space is limited or when transferring data over a secure tunnel.
Sources: README.md:113-118, cmd/pmm-dump/main.go:110, cmd/pmm-dump/util.go:415-420
Local Development and Testing
The project includes a Makefile for managing a local testing environment using Docker.
| Command | Description |
|---|---|
make up | Starts PMM Server, Client, and MongoDB containers. |
make run-tests | Executes all unit and end-to-end tests. |
make down | Shuts down and removes all local test containers. |
End-to-end tests are located in internal/test/e2e and require Docker Compose v2. These tests verify the full export/import cycle, data integrity (validation), and security features like credential redaction in logs.
Sources: README.md:155-175, internal/test/README.md:3-12, cmd/pmm-dump/security_test.go:1-10