Prometheus Adapter
Relevant source files
The following files were used as context for generating this wiki page:
The Prometheus Adapter connects Prometheus monitoring data to the Infrahub ecosystem. It retrieves metric samples from Prometheus endpoints and transforms them into structured models for synchronization. The adapter supports two primary modes: scraping raw text exposition formats from exporters and querying the Prometheus HTTP API using PromQL.
This adapter functions within the larger Sync architecture by acting as a source. It leverages the DiffSyncMixin to participate in the reconciliation process, loading metrics into an in-memory store that can be compared against a destination system like Infrahub.
Sources: infrahub_sync/adapters/prometheus.py:270-282, dev/knowledge/sync-architecture.md:1-12
Architecture and Components
The Prometheus Adapter consists of specialized clients for data retrieval, a lookup resolver for cross-metric data joining, and the core adapter class that interfaces with the sync engine.
Data Retrieval Clients
The adapter uses two distinct client classes to fetch data based on the configured mode:
- PrometheusScrapeClient: Connects to text exposition endpoints (e.g.,
/metrics) and parses them into normalized sample dictionaries. - PrometheusAPIClient: Interfaces with the Prometheus HTTP API (e.g.,
/api/v1/query) to execute instant queries. This allows the use of PromQL to join or filter data before it enters the sync pipeline.
Sources: infrahub_sync/adapters/prometheus.py:53-65, infrahub_sync/adapters/prometheus.py:91-105
Data Flow Diagram
The following diagram illustrates how the adapter retrieves and processes Prometheus data.
The adapter normalizes external data into a consistent "sample" shape before processing.
Sources: infrahub_sync/adapters/prometheus.py:165-177, infrahub_sync/adapters/prometheus.py:339-366
Configuration
The adapter is configured via the source section of a config.yml file. It requires a URL and supports optional authentication and mode settings.
Settings Table
| Setting | Type | Default | Description |
|---|---|---|---|
mode | string | scrape | Data retrieval mode: scrape or api. |
url | string | N/A | Base URL of the Prometheus server or exporter. |
endpoint | string | /metrics or /api/v1/query | Path to the specific endpoint. |
auth_method | string | none | Authentication type: none, basic, or bearer. |
timeout | float | 10 | Request timeout in seconds. |
verify_ssl | boolean | True | Whether to verify SSL certificates. |
promql.resources | dict | {} | (API mode only) Mapping of model names to PromQL queries. |
Sources: infrahub_sync/adapters/prometheus.py:284-337, examples/prometheus_to_infrahub (node_exporter)/config.yml:3-9
Environment Variables
The adapter prioritizes environment variables for sensitive credentials:
PROM_URLorPROM_ADDRESS: The target URL.PROM_USERNAME: Username for basic authentication.PROM_PASSWORD: Password for basic authentication.PROM_TOKEN: Token for bearer authentication.
Sources: infrahub_sync/adapters/prometheus.py:299-307
Model Loading and Mapping
The model_loader method orchestrates the conversion of raw samples into DiffSyncModel instances. It uses the schema_mapping defined in the configuration to identify which metrics correspond to which destination models.
Field Mapping and Lookups
Users map Prometheus data to destination fields using standard paths (e.g., labels.nodename) or specialized lookup functions.
- Standard Mapping: Extracts values from labels or metric metadata.
- Lookup Function: The
lookup(metric, key_path, value_path, default?)syntax allows retrieving values from different metrics by joining on a common label or path.
Sources: infrahub_sync/adapters/prometheus.py:417-430, dev/knowledge/schema-mapping.md:20-30
Example Configuration Snippet
schema_mapping:
- name: VirtualizationVirtualMachine
mapping: node_uname_info
identifiers: ["name"]
fields:
- name: name
mapping: labels.nodename
transforms:
- field: mem_total_t
expression: "{{ lookup('node_memory_MemTotal_bytes', {}, 'value') or 0 }}"
Sources: examples/prometheus_to_infrahub (node_exporter)/config.yml:25-50
Data Transformation Logic
The adapter utilizes LookupResolver to handle complex relationships between metrics that are not inherently joined in Prometheus.
Sequence of Resolution
The sequence demonstrates how data is cached and indexed during a single sync run to optimize lookups.
Sources: infrahub_sync/adapters/prometheus.py:339-366, infrahub_sync/adapters/prometheus.py:389-415
Transformation Filters
The adapter inherits standard filtering capabilities from DiffSyncModelMixin. It applies filter_records and transform_records to the metrics before creating DiffSync items, ensuring only relevant data is synchronized.
Sources: infrahub_sync/adapters/prometheus.py:396-400, dev/knowledge/adapter-anatomy.md:38-45
Summary
The Prometheus Adapter provides a flexible bridge for importing monitoring data into Infrahub. By supporting both raw scraping and PromQL-based API queries, it accommodates various deployment scenarios. The inclusion of the LookupResolver enables users to reconstruct complex infrastructure relationships from disparate metric streams directly within the YAML configuration.