PostgreSQL Integration

Relevant source files

The following files were used as context for generating this wiki page:

• pkg/postgres/source.go
• cmd/pmm-dump/main.go
• cmd/pmm-dump/util.go
• cmd/pmm-dump/util_test.go
• README.md

The PostgreSQL integration in pmm-dump is designed to facilitate the export and import of internal PMM Server databases. This module ensures that critical configuration and metadata stored within PostgreSQL—specifically for components like Grafana—can be bundled into the tool's compressed dump files alongside metrics and Query Analytics (QAN) data.

The integration operates as a specialized data source that interacts with the PostgreSQL instance via standard system utilities. It primary targets a predefined set of databases: grafana, ssmDB, and percona.

Sources: README.md:1-5, cmd/pmm-dump/util.go:378-382

Architecture and Components

The PostgreSQL integration is built around the Source struct, which implements the generic dump.Source interface. This allows the core transferer logic to treat PostgreSQL data as a series of chunks similar to VictoriaMetrics or ClickHouse data.

Data Source Implementation

The implementation leverages external binaries (pg_dump and pg_restore) to perform data operations. This ensures compatibility with PostgreSQL's custom format and handles schema-specific complexities.

The diagram shows how the PostgreSQL Source interfaces with the system's underlying PostgreSQL utilities to handle data transfer.

Sources: pkg/postgres/source.go:28-32, pkg/postgres/source.go:44-50, pkg/postgres/source.go:70-75

Configuration Structure

The integration requires a configuration object that defines the connection parameters and the specific databases to be processed.

Field	Type	Description
ConnectionURL	string	The full connection string (e.g., postgres://user:pass@host:port/db).
Databases	[]string	List of database names to include in the dump. Defaults to grafana, ssmDB, and percona.

Sources: pkg/postgres/source.go:35-37, cmd/pmm-dump/util.go:378-382

Data Export Logic

The export process is handled by the ReadChunk function. Unlike time-series data which is split by time ranges, PostgreSQL chunks are organized by database index.

Export Workflow

1. Database Selection: The system determines the database name based on the ChunkMeta.Index.
2. Command Execution: It executes pg_dump with the --data-only and --format=custom flags.
3. Schema Filtering: The --schema flag is used to target specific databases from the configuration.
4. Chunk Creation: The output bytes are wrapped into a dump.Chunk object with a .pgdump extension.

The sequence diagram illustrates the synchronous execution of the system dump utility during the export phase.

Sources: pkg/postgres/source.go:40-62

Data Import Logic

The import process utilizes the WriteChunk function to restore data into a target PMM instance.

Restore Process

The tool uses pg_restore to ingest data from the dump file. Key flags used include:

• --clean: Drops database objects before recreating them.
• --if-exists: Prevents errors during the clean phase if objects are missing.
• --dbname: Directs the data to the specified PostgreSQL connection string.

If pg_restore finishes with warnings (exit code 1), the tool logs these as warnings rather than fatal errors to account for common non-critical restoration messages.

Sources: pkg/postgres/source.go:67-84

CLI Integration and Configuration

The PostgreSQL integration is managed via several command-line flags and environment variables.

Global Flags

Flag	Environment Variable	Default	Description
--postgres-url	PMM_POSTGRES_URL	N/A	PostgreSQL connection string.
--dump-postgres	PMM_DUMP_POSTGRES	true	Whether to include PostgreSQL data in the dump.

Initialization Logic

The preparePostgresSource function in the utility package validates if the source should be active. It returns ok: false if --dump-postgres is disabled or if the connection URL is empty.

func preparePostgresSource(dumpPostgres bool, url string) (*postgres.Source, bool) {
	if !dumpPostgres || url == "" {
		return nil, false
	}

	c := &postgres.Config{
		ConnectionURL: url,
		Databases:     []string{"grafana", "ssmDB", "percona"},
	}

	log.Debug().Msgf("Got PostgreSQL URL: %s", c.ConnectionURL)

	return postgres.NewSource(*c), true
}

Sources: cmd/pmm-dump/main.go:58-64, cmd/pmm-dump/util.go:375-387, cmd/pmm-dump/util_test.go:53-68

Summary

PostgreSQL integration is a core component of pmm-dump that handles the "meta" side of PMM data. By wrapping pg_dump and pg_restore, it provides a reliable way to migrate Grafana dashboards, user settings, and internal PMM metadata. The system automatically targets the three primary databases used by PMM and integrates seamlessly into the worker-based transfer architecture of the application.

Sources: pkg/postgres/source.go:28-85, cmd/pmm-dump/main.go:210-213