Log Integration & Capturing
Relevant source files
The following files were used as context for generating this wiki page:
Logfire provides a robust system for integrating with standard Python logging libraries and capturing log records as OpenTelemetry-compatible signals. By wrapping OpenTelemetry's logging SDK, Logfire allows developers to funnel logs from diverse sources—including the standard logging module, loguru, and structlog—into a unified observability backend.
The system is built around a ProxyLoggerProvider that manages the transition from raw log data to structured telemetry, ensuring that log levels, message templates, and contextual attributes are preserved and correctly mapped to the OpenTelemetry data model. Sources: logfire/_internal/logs.py, logfire/_internal/config.py:440-450
Architecture Overview
The log integration architecture is designed to intercept log calls at their source and translate them into Logfire events. This is achieved through specific handlers and processors tailored for different logging frameworks.
The diagram shows how various logging frameworks feed into specific Logfire adapters before reaching the core ProxyLoggerProvider. Sources: logfire/integrations/logging.py, logfire/integrations/loguru.py, logfire/integrations/structlog.py
Standard Logging Integration
Logfire integrates with Python's built-in logging module via the LogfireLoggingHandler. This handler translates logging.LogRecord objects into Logfire logs.
Mapping Log Levels
Logfire maps standard logging levels to numerical values used within its internal processing engine.
| Standard Level | Numerical Value |
|---|---|
| NOTSET | 0 |
| DEBUG | 5 |
| INFO | 9 |
| WARNING / WARN | 13 |
| ERROR | 17 |
| CRITICAL / FATAL | 21 |
Sources: logfire/_internal/constants.py, logfire/_internal/config.py:108 (Derived from LEVEL_NUMBERS)
The LogfireLoggingHandler
The LogfireLoggingHandler is responsible for extracting information from LogRecord instances, including:
- Attributes: Arbitrary data passed in the
extradictionary. - Exc Info: Exception tracebacks if present.
- Message Templates: Formatting the message using the record's arguments.
Sources: logfire/integrations/logging.py, tests/test_logfire_api.py:175
Third-Party Framework Support
Loguru Integration
Logfire supports loguru through a specialized handler. This integration ensures that loguru's rich contextual information and structured data are captured. The logfire.loguru_handler() function provides a compatible sink for loguru.logger.add().
Sources: logfire/integrations/loguru.py, tests/test_logfire_api.py:178
Structlog Integration
For structlog, Logfire provides a StructlogProcessor. This processor acts as a middleware in the structlog pipeline, converting structured event dicts into Logfire log calls.
The sequence shows how StructlogProcessor intercepts structured logs and forwards them to Logfire. Sources: logfire/integrations/structlog.py, tests/test_logfire_api.py:181
Internal Log Processing
The ProxyLoggerProvider serves as the primary interface between the integrations and the OpenTelemetry SDK.
ProxyLoggerProvider Logic
The provider maintains a reference to an underlying OpenTelemetry LoggerProvider. When configure() is called, the ProxyLoggerProvider is updated from a no-op state to a functional SDK provider. It also handles "minimum level" filtering, ensuring that logs below a certain threshold are discarded before reaching the export pipeline.
Key methods include:
get_logger(): Returns a logger instance associated with a specific instrumentation scope.set_provider(): Transitions from No-Op to a real SDK provider.set_min_level(): Configures the threshold for log capturing.
Sources: logfire/_internal/logs.py, logfire/_internal/config.py:440-520
Log Capture Flow
The flow describes the path a log record takes through filtering, scrubbing, and multi-processing before final export. Sources: logfire/_internal/config.py:530-550, logfire/_internal/exporters/logs.py
Configuration and Management
Logging behavior is heavily influenced by the global configuration set during logfire.configure().
| Option | Type | Description |
|---|---|---|
min_level | int | LevelName | Minimum level for logs to be created. |
console | ConsoleOptions | Controls if/how logs are printed to the terminal. |
scrubbing | ScrubbingOptions | Defines patterns for redacting sensitive data in logs. |
Sources: logfire/_internal/config.py:146-240
Console Output
When console output is enabled, Logfire uses a ConsoleLogExporter to format log records for terminal display, often including timestamps, tags, and stylized levels.
Sources: logfire/_internal/config.py:118-135, logfire/_internal/exporters/console.py
Conclusion
Log Integration & Capturing in Logfire is a multi-layered system that bridges the gap between traditional Python logging and modern structured telemetry. By providing dedicated adapters for logging, loguru, and structlog, and routing them through a central ProxyLoggerProvider, Logfire ensures consistent observability across different parts of an application while maintaining high performance through early-stage filtering and efficient processing.