PMM Agent

Relevant source files

The following files were used as context for generating this wiki page:

• agent/agents/supervisor/supervisor_test.go
• api/inventory/v1/agents.proto
• CONTRIBUTING.md
• managed/models/database.go
• api/management/v1/json/v1.json
• admin/commands/summary.go

PMM Agent is a core component of the Percona Monitoring and Management (PMM) Client. It acts as a central supervisor on monitored nodes, responsible for managing the lifecycle of various exporters and collectors, such as node_exporter, mysqld_exporter, and Query Analytics (QAN) agents. It maintains a persistent connection to the PMM Server (pmm-managed) to receive configuration updates and report the status of the agents it supervises.

Sources: CONTRIBUTING.md, agent/agents/supervisor/supervisor_test.go:61-75

Architecture and Supervision

The PMM Agent operates primarily through a Supervisor component. The Supervisor is responsible for translating the desired state received from the PMM Server into running processes or built-in agents on the local node. It handles starting, stopping, and restarting agents based on configuration changes or failures.

Agent Lifecycle Management

The Supervisor manages two main categories of agents:

1. Agent Processes: External binary processes like Prometheus exporters (e.g., node_exporter, postgres_exporter).
2. Built-in Agents: Logic integrated directly into the PMM Agent (often used for testing or specific internal tasks).

The following diagram illustrates the supervision flow where the PMM Agent receives a state request and synchronizes local processes:

Supervisor State Transitions

The Supervisor monitors the health of its sub-processes. When an agent process fails to start or exits unexpectedly, the Supervisor transitions it through various statuses.

Status	Description
AGENT_STATUS_STARTING	The process is being initialized.
AGENT_STATUS_RUNNING	The process is active and healthy.
AGENT_STATUS_STOPPING	A request to terminate the process is being handled.
AGENT_STATUS_DONE	The process has successfully stopped.
AGENT_STATUS_WAITING	The process encountered an error and is waiting for a retry.
AGENT_STATUS_INITIALIZATION_ERROR	Failed to start due to bad configuration or missing binaries.

Sources: agent/agents/supervisor/supervisor_test.go:65-75, api/inventory/v1/agents.proto:700-720

Supported Agent Types

PMM Agent supports a wide variety of database and system monitoring agents. These are defined in the project's inventory API and mapped to specific exporter binaries or internal modules.

Agent Type	Purpose
AGENT_TYPE_NODE_EXPORTER	Exports machine-level metrics (CPU, Memory, Disk).
AGENT_TYPE_MYSQLD_EXPORTER	Scrapes metrics from MySQL/Percona Server.
AGENT_TYPE_POSTGRES_EXPORTER	Scrapes metrics from PostgreSQL instances.
AGENT_TYPE_VM_AGENT	Collects data from exporters and pushes to VictoriaMetrics.
AGENT_TYPE_QAN_MYSQL_PERFSCHEMA_AGENT	Collects Query Analytics data via MySQL Performance Schema.
AGENT_TYPE_NOMAD_AGENT	Orchestrates PMM Client components via Nomad.

Sources: api/inventory/v1/agents.proto:23-50, CONTRIBUTING.md

Data Model and Persistence

The PMM Agent's configuration and the metadata of the agents it runs are stored within the PMM Server's database (PostgreSQL). The agents table tracks every instance managed by a PMM Agent.

Database Schema (Agents Table)

The schema defines relationships between nodes, services, and the agents themselves.

Key constraints in the data model ensure that:

• A PMMAgent runs on a Node.
• An exporter agent (like MySQLdExporter) is linked to a PMMAgent via pmm_agent_id.
• Agents are linked to either a node_id or a service_id to provide monitoring context.

Sources: managed/models/database.go:145-148, managed/models/database.go:182-186

Diagnostics and Summary

For troubleshooting, the PMM system provides tools to aggregate PMM Agent logs and state. The pmm-admin summary command interacts with the local PMM Agent API to generate a diagnostic archive.

Summary Data Collection Flow

When a user requests a summary, the client tool performs several actions:

1. Local Status: Retrieves JSON status from the PMM Agent (including network info and agent lists).
2. Logs: Downloads a logs.zip directly from the PMM Agent's local web server (typically port 42000).
3. VMAgent Targets: If vmagent is running, it queries its internal targets API to see what is being scraped.
4. Profiling: Optionally collects Go pprof data (heap, CPU traces) from the pmm-agent process.

Sources: admin/commands/summary.go:95-135, admin/commands/summary.go:171-200

Conclusion

The PMM Agent is the foundational service of the PMM Client suite. By implementing a robust supervision model, it ensures that monitoring exporters remain running and correctly configured according to the PMM Server's requirements. Its tight integration with the project's inventory API and local diagnostic tools makes it a critical component for both real-time monitoring and system troubleshooting.