Metrics Collection
Relevant source files
The following files were used as context for generating this wiki page:
Pydantic Logfire provides a robust metrics collection system built on top of OpenTelemetry (OTel). It supports standard OTel metric instruments including counters, gauges, and histograms, while offering opinionated defaults such as Delta aggregation for improved resolution and reduced payload sizes. The system is designed to collect both application-specific business logic metrics and infrastructure-level system metrics.
Architecture and Infrastructure
The metrics infrastructure centered around the ProxyMeterProvider, which acts as a wrapper for the underlying OpenTelemetry MeterProvider. This allows Logfire to manage metric collection lifecycles, including initialization, reconfiguration, and shutdown, without directly exposing the complexities of OTel's global state to the user.
Metrics Data Flow
The flow of metrics data moves from the application instruments through a reader and exporter to the Logfire backend.
Configuration and Options
Metrics are configured through MetricsOptions, which allows for customization of readers, views, and experimental features.
| Option | Type | Description |
|---|---|---|
additional_readers | Sequence[MetricReader] | Custom OTel metric readers to use alongside the default. |
collect_in_spans | bool | Experimental: Adds counter and histogram values to active spans. |
views | Sequence[View] | Custom OTel views to apply during collection. |
preferred_temporality | dict | Maps instrument types to AggregationTemporality.DELTA or CUMULATIVE. |
Sources: logfire/_internal/config.py:228-261, logfire/_internal/config.py:91-101
Metric Instruments
Logfire supports several types of metric instruments, each suitable for different observation use cases. By default, Logfire prefers AggregationTemporality.DELTA for counters and histograms to provide better time-series resolution.
Application Metrics
The following table describes the primary instruments available via the Logfire API:
| Instrument | Method | Behavior |
|---|---|---|
| Counter | metric_counter | Monotonic; values only increase. Uses Delta aggregation by default. |
| UpDownCounter | metric_up_down_counter | Non-monotonic; can increase or decrease. |
| Gauge | metric_gauge | Records a value at a specific point in time; overwrites previous values. |
| Histogram | metric_histogram | Records the distribution of values. Uses Exponential Bucket aggregation. |
Sources: tests/test_metrics.py:44-150, logfire/_internal/config.py:91-101
Aggregation Temporality
Logfire configures a specific mapping for preferred_temporality to ensure data consistency in the backend:
- DELTA:
Counter,Histogram,ObservableCounter - CUMULATIVE:
UpDownCounter,ObservableGauge,ObservableUpDownCounter
Sources: logfire/_internal/config.py:91-101, tests/test_metrics.py:76-80
System Metrics Collection
System metrics provide visibility into host and process-level performance. These are not collected by default but can be enabled via logfire.instrument_system_metrics().
Collection Scopes
Logfire defines several "base" configurations for system metrics:
- Basic: Includes CPU utilization, memory availability, and swap usage.
- Full: Includes extensive networking (IO, packets, errors), disk (IO, time), and detailed CPython garbage collection stats.
Implementation Detail: Quiet Exporter
To prevent telemetry loops and console noise, Logfire uses a QuietMetricExporter. This wrapper suppresses internal errors (such as ConnectionError) during the export process to ensure that failures in the observability pipeline do not crash the primary application.
Sources: logfire/_internal/exporters/quiet_metrics.py, tests/test_metrics.py:317-347
Metrics in Spans (Experimental)
Logfire features an experimental capability to aggregate metric values directly into active spans. When collect_in_spans=True is configured, calls to increment counters or record histograms that occur within a logfire.span context will have those values attached to the span attributes under the logfire.metrics key.
# Example of metrics aggregation in spans
with logfire.span('process_request'):
tokens_counter.add(150, attributes={'model': 'gpt-4'})
# The span will eventually contain the 'tokens' total in its attributes
Sources: logfire/_internal/config.py:252-253, tests/test_metrics.py:350-394
Summary
Metrics collection in Logfire is a specialized implementation of OpenTelemetry designed for ease of use and high-performance data export. By defaulting to Delta aggregation and providing pre-configured system metric sets, it allows developers to gain deep insights into both application performance and infrastructure health with minimal boilerplate. All metrics are exported via OTLP/HTTP using a robust, "quiet" pipeline to ensure application stability.