Introduction to Pydantic Logfire
Relevant source files
The following files were used as context for generating this wiki page:
Pydantic Logfire is an observability platform designed to provide simple yet powerful insights into application behavior. Built by the team behind Pydantic Validation, it functions as an opinionated wrapper around OpenTelemetry, supporting the full spectrum of telemetry signals including traces, metrics, and logs. While the SDKs for Python, Rust, and TypeScript are open-source, the core platform for recording and displaying data remains a closed-source service available via cloud or enterprise licensing.
The system is specifically optimized for Python, offering rich display of Python objects, event-loop telemetry, and native integration with Pydantic models to analyze data validation flows. It leverages standard SQL for data querying, allowing developers to interact with telemetry data using familiar tools and BI integrations.
Sources: README.md:7-24, CLAUDE.md:1-8, pyproject.toml:9-11
Core Architecture and Components
Logfire's architecture is built on a "Proxy" pattern where internal components wrap standard OpenTelemetry (OTel) providers. This allows Logfire to manage configuration and state while remaining compatible with the broader OTel ecosystem.
Component Overview
| Component | Responsibility | Source Reference |
|---|---|---|
LogfireConfig | Centralized management of credentials, exporters, and sampling logic. | logfire/_internal/config.py:388 |
ProxyTracerProvider | A wrapper around OTel TracerProvider that allows for deferred initialization. | logfire/_internal/config.py:539 |
ProxyMeterProvider | Handles metrics collection and exports via OTLP. | logfire/_internal/config.py:542 |
ProxyLoggerProvider | Manages structured logging and correlates logs with active spans. | logfire/_internal/config.py:544 |
LogfireClient | Communicates with the Logfire backend for project and user management. | logfire/_internal/cli/init.py:67 |
System Data Flow
The following diagram illustrates how telemetry data flows from the user application through the Logfire SDK to the backend.
The SDK intercept calls via its Proxy providers, processes them through various processors (batching, sampling, scrubbing), and exports them using the OTLP protocol.
Sources: logfire/_internal/config.py:539-688, CLAUDE.md:14-25
Configuration and Initialization
The primary entry point for the SDK is the logfire.configure() function. This function populates a global LogfireConfig object which governs how telemetry is collected and where it is sent.
Configuration Options
| Option | Type | Description |
|---|---|---|
token | str | list[str] | The project write token(s) used for authentication. |
service_name | str | Identifying name for the service in the dashboard. |
send_to_logfire | bool | str | Whether to export data to the Logfire backend. |
console | ConsoleOptions | Settings for local terminal output of spans and logs. |
scrubbing | ScrubbingOptions | Configuration for redacting sensitive data. |
sampling | SamplingOptions | Controls for head and tail-based sampling. |
Sources: logfire/_internal/config.py:277-380, logfire/_internal/config.py:401-440
Initialization Sequence
When configure() is called, the SDK performs a multi-step setup to ensure the environment is ready for telemetry.
CLI and Tooling
Logfire includes a comprehensive CLI for managing projects, authentication, and local execution. The CLI is accessible via the logfire command.
Key CLI Commands
logfire auth: Handles device-based authentication to obtain user tokens.logfire projects: Subcommands tolist,use, or createnewprojects.logfire run: Executes a Python script or module with automatic instrumentation enabled.logfire inspect: Analyzes the current environment and recommends missing OTel instrumentation packages.logfire whoami: Displays the currently authenticated user and active project.
Sources: logfire/_internal/cli/init.py:228-305, logfire/_internal/cli/run.py:69-105
Automatic Instrumentation Logic
The logfire run command uses a mapping to automatically load OTel instrumentation for installed libraries.
The CLI maintains an OTEL_INSTRUMENTATION_MAP that links library names (e.g., fastapi, requests) to their respective Logfire instrumentation functions.
Sources: logfire/_internal/cli/run.py:15-58, logfire/_internal/cli/run.py:118-142
Integration with Pydantic
A unique feature of Logfire is its deep integration with Pydantic. It provides a dedicated plugin that intercepts Pydantic validation events to provide insights into data structures and validation failures.
- Analytics: Built-in analytics on validation performance and error rates.
- Data Flow: Visibility into the exact data objects passing through models.
- Modes: Support for different recording modes including
all(every validation),failure(only errors), andmetrics(aggregated counts only).
Sources: README.md:19-21, logfire/_internal/config.py:183-201, pyproject.toml:79
Conclusion
Pydantic Logfire provides a developer-friendly abstraction over OpenTelemetry, emphasizing ease of use without sacrificing the depth of telemetry. By combining structured logging, distributed tracing, and metric collection with a SQL-queryable backend and native Pydantic support, it serves as a comprehensive observability solution for modern Python applications. Its hybrid open-core model allows developers to use standard OTel protocols while benefiting from a specialized UI and advanced Python-centric features.
Sources: README.md:7-13, CLAUDE.md:1-5