Metadata Management
Relevant source files
The following files were used as context for generating this wiki page:
Metadata Management in pmm-dump is responsible for capturing, storing, and displaying critical information about the state of a PMM (Percona Monitoring and Management) Server at the time of export. This metadata ensures that data dumps are portable, traceable, and correctly interpreted during the import process on a different PMM instance.
The system captures technical details such as the PMM server version, timezone configurations, the exact command-line arguments used for the export, and information about the monitored services. This information is stored in a dedicated meta.json file within the dump archive.
Sources: pkg/dump/dump.go:21-26, README.md:105-108
Metadata Structure
The core of metadata management is the Meta struct, which defines the schema for the meta.json file. It includes versioning for the pmm-dump tool itself, as well as state information from the source PMM server.
Data Model
| Field | Type | Description |
|---|---|---|
version | PMMDumpVersion | Contains Git branch and commit of the tool used. |
pmm-server-version | string | The version of the PMM Server from which data was exported. |
max_chunk_size | int64 | The maximum size of data chunks. |
pmm-server-timezone | *string | Preferred timezone in PMM-server Grafana settings. |
arguments | string | Redacted string of CLI arguments used for the export. |
vm-data-format | string | Format of VictoriaMetrics data (e.g., "json" or "native"). |
pmm-server-services | []PMMServerService | (Optional) List of monitored services and their IDs. |
Sources: pkg/dump/dump.go:30-47
Service Information
When the --export-services-info flag is provided, the tool retrieves a detailed list of monitored services. Each service entry includes the service name, Node ID, Node Name, and associated Agent IDs.
Sources: cmd/pmm-dump/util.go:102-140, pkg/dump/dump.go:42-47
Architecture and Data Flow
The metadata management process follows a clear flow from collection during export to retrieval during import or manual inspection via the show-meta command.
The diagram shows how metadata is generated during export, utilized to configure the environment during import, and accessed via management commands.
Sources: cmd/pmm-dump/main.go:216-224, cmd/pmm-dump/util.go:183-238
Metadata Collection Logic
Metadata is composed using the composeMeta function, which orchestrates several API calls to the PMM Server to gather the required information.
Version and Timezone Retrieval
The system identifies the PMM version via the /v1/server/version endpoint. The timezone is retrieved from the Grafana organization preferences at /graph/api/org/preferences. If the timezone is set to "browser" or is empty, it is treated as nil in the metadata.
Sources: cmd/pmm-dump/util.go:72-100, cmd/pmm-dump/util.go:167-181
Argument Redaction
To prevent sensitive information from being stored in the metadata, a redaction mechanism is applied to the CLI arguments. URLs containing credentials (e.g., --pmm-url, --click-house-url) have their usernames and passwords replaced with ***:***.
func redactURL(raw string) string {
if !strings.Contains(raw, "@") {
return raw
}
if !urlRedactRe.MatchString(raw) {
return "***"
}
return urlRedactRe.ReplaceAllString(raw, "${1}***:***@")
}
Sources: cmd/pmm-dump/util.go:202-214, cmd/pmm-dump/util.go:429-437
VictoriaMetrics Data Format Management
A critical piece of metadata is the vm-data-format. VictoriaMetrics supports both a standard JSON format and a "native" export format. While the native format is smaller and more efficient, it may be incompatible between different PMM versions. The metadata tracks which format was used to ensure the importer uses the correct API endpoint (/api/v1/import vs /api/v1/import/native).
| Format | CLI Flag | Meta Value | API Endpoint |
|---|---|---|---|
| Native | --vm-native-data | native | /api/v1/export/native |
| JSON | (Default) | json | /api/v1/export |
Sources: pkg/victoriametrics/source.go:73-76, cmd/pmm-dump/main.go:275-300
Meta Management Commands
The tool provides specific functionality for interacting with metadata through the show-meta command.
show-meta
This command allows users to inspect the contents of a dump's meta.json without performing a full import. It supports two modes:
- Prettified (Default): Prints formatted build info, PMM version, chunk sizes, and a list of services.
- Raw: Outputs the raw JSON structure of the metadata.
Sources: cmd/pmm-dump/main.go:343-375
Conclusion
Metadata Management serves as the configuration backbone for pmm-dump. By encapsulating the source server's state, tool versioning, and redacted execution parameters into a structured meta.json file, it enables consistent data transfers and provides an audit trail for troubleshooting PMM performance data migrations.