Exporters and Span Processors
Relevant source files
The following files were used as context for generating this wiki page:
Exporters and Span Processors form the data egress pipeline of the Logfire SDK. Built upon the OpenTelemetry Python SDK, these components are responsible for intercepting spans and log records as they are created, processing them (e.g., buffering, filtering, or modifying), and transmitting them to various backends such as the Logfire API or the local console.
The architecture uses an opinionated set of wrappers to provide high reliability, including features like automatic retry with exponential backoff, suppression of internal telemetry to prevent infinite loops, and specialized handling for "pending spans" to provide real-time visibility into long-running operations.
Sources: logfire/_internal/config.py:28-60, CLAUDE.md
Core Architecture and Data Flow
The pipeline operates by wrapping standard OpenTelemetry processors and exporters in specialized Logfire components. When a span is created or ended, it passes through a chain of processors before reaching an exporter.
The Export Pipeline
This diagram illustrates the hierarchical wrapping of components. The CheckSuppressInstrumentationProcessorWrapper ensures that telemetry generated by Logfire's own exporters doesn't create a recursive loop.
Sources: logfire/_internal/config.py:650-750, tests/test_configure.py:1015-1030
Span Processors
Span processors are responsible for the lifecycle of a span. Logfire implements several custom processors to enhance the standard OpenTelemetry behavior.
Dynamic Batching
The DynamicBatchSpanProcessor is a specialized version of the standard BatchSpanProcessor. It manages a background thread that exports batches of spans. A key feature is its ability to adjust the export delay dynamically: it starts with a short delay (e.g., 100ms) for the first few spans to provide immediate feedback, then increases to a longer default (e.g., 500ms) to improve throughput and reduce overhead.
Sources: logfire/_internal/exporters/dynamic_batch.py, tests/test_configure.py:220-260
Pending Span Management
Logfire introduces the concept of "pending spans"—telemetry sent when a span starts rather than just when it ends.
PendingSpanProcessor: Intercepts the start of a span and exports a "pending" version of it.RemovePendingSpansExporter: This exporter wrapper filters out spans marked aspending_spanbefore they are sent to backends that do not support them, or ensures they are handled correctly in the primary Logfire pipeline.
Sources: logfire/_internal/config.py:755-765, logfire/_internal/exporters/remove_pending.py
Exporters
Exporters are the final stage of the pipeline, handling the actual transmission of data.
OTLP Exporter Stack
The primary exporter for the Logfire backend uses the OTLP (OpenTelemetry Line Protocol) over HTTP. It is typically configured with several layers:
| Layer | Class | Purpose |
|---|---|---|
| Transport | OTLPSpanExporter | Base OpenTelemetry class for OTLP/HTTP. |
| Reliability | RetryFewerSpansSpanExporter | Retries failed exports with smaller batch sizes if the backend returns "Entity Too Large" or similar errors. |
| Silence | QuietSpanExporter | Suppresses standard library logging/warnings during network failures to prevent console noise. |
| Validation | BodySizeCheckingOTLPSpanExporter | Prevents sending payloads that exceed backend limits. |
Sources: logfire/_internal/exporters/otlp.py, logfire/_internal/config.py:710-740
Console Exporters
For local development, Logfire provides highly formatted console output.
SimpleConsoleSpanExporter: A flat list of logs and spans.IndentedConsoleSpanExporter: Uses indentation to visualize the span hierarchy.ShowParentsConsoleSpanExporter: The default style; it prints parent span names above child logs even if the parent was previously printed, providing context for interleaved logs.
Sources: logfire/_internal/exporters/console.py, logfire/_internal/config.py:110-130
Configuration Elements
The export behavior is primarily controlled via logfire.configure().
| Option | Type | Description |
|---|---|---|
send_to_logfire | bool | str | Enables the OTLP pipeline. Can be 'if-token-present'. |
console | ConsoleOptions | Configures colors, styles (simple, indented, show-parents), and timestamps. |
additional_span_processors | Sequence | Allows users to inject custom OTel processors into the pipeline. |
advanced.log_record_processors | Sequence | Configures custom processors specifically for log records. |
Sources: logfire/_internal/config.py:410-550, tests/test_configure.py:310-350
Internal Suppression Mechanism
To prevent infinite recursion, Logfire uses a context-local suppression flag. The CheckSuppressInstrumentationProcessorWrapper (and its log counterpart) checks this flag before allowing any data through.
This ensures that network errors occurring inside the OTLPLogExporter do not trigger new log records that would again fail and attempt to export, causing a crash or resource exhaustion.
Sources: logfire/_internal/config.py:650-660, logfire/_internal/exporters/processor_wrapper.py, logfire/_internal/utils.py:suppress_instrumentation
Conclusion
The Exporters and Span Processors system in Logfire provides a robust, production-ready pipeline for telemetry. By wrapping standard OpenTelemetry components, it adds essential features for distributed systems such as dynamic batching, intelligent retries, and real-time pending span visibility while maintaining full compatibility with the OTel ecosystem.
Summary of significance:
- Reliability: Layered retry and payload-size management.
- Observability: Real-time "pending" status for long-running spans.
- Clarity: Context-aware console exporters for local debugging.
- Safety: Suppression wrappers to prevent telemetry recursion.
Sources: README.md, logfire/_internal/config.py