Sampling Strategies

Relevant source files

The following files were used as context for generating this wiki page:

• logfire/sampling/__init__.py
• logfire/sampling/_tail_sampling.py
• logfire/_internal/config.py
• tests/test_tail_sampling.py
• tests/test_configure.py
• mkdocs.yml

Sampling strategies in Logfire determine which spans and logs are recorded and exported to the backend. By intelligently filtering telemetry data, applications can maintain high observability for critical events (like errors or slow requests) while reducing the volume and cost associated with high-traffic, routine operations.

Logfire implements a multi-stage sampling pipeline that includes traditional "Head Sampling" (deciding at the start of a trace) and more advanced "Tail Sampling" (deciding after a span has completed based on its outcome). This hybrid approach ensures that interesting traces are preserved even if they would otherwise be dropped by a simple random sampler.
Sources: logfire/_internal/config.py:465-470, logfire/sampling/_tail_sampling.py:12-25

Architecture and Data Flow

The sampling logic is integrated into the OpenTelemetry pipeline via custom Sampler implementations and SpanProcessor wrappers. The flow begins when a span is started and continues through its lifecycle until it is exported or dropped.

This diagram shows the path of a span through the Head and Tail sampling stages.
Sources: logfire/_internal/config.py:720-745, logfire/sampling/_tail_sampling.py:55-70

Head Sampling

Head sampling is the first stage of the process, occurring as soon as a span is created. It is primarily used to control the overall volume of data by randomly selecting a percentage of traces to record.

Configuration

Head sampling is configured via the SamplingOptions class, specifically the head parameter. It supports several types of values:

• Float/Int: A value between 0.0 and 1.0 representing the ratio of traces to keep.
• Sampler Instance: A custom OpenTelemetry Sampler for advanced logic.
• Boolean: True (Always On) or False (Always Off).

Sources: logfire/sampling/init.py:20-35, logfire/_internal/config.py:722-728

Logic and Interaction

If the head sampler decides to drop a trace, no data for that trace is recorded unless forced by subsequent logic. When using SamplingOptions.level_or_duration, the head sampler acts as a "background rate" for routine logs, while high-level logs or long-duration spans bypass this random selection to ensure they are captured.
Sources: logfire/sampling/init.py:60-80, tests/test_tail_sampling.py:200-215

Tail Sampling

Tail sampling occurs when a span ends. This allows Logfire to make sampling decisions based on attributes that are only known after execution, such as the final log level, the duration of the span, or the presence of an exception.

The TailSamplingProcessor

The TailSamplingProcessor is a wrapper around a SynchronousMultiSpanProcessor. It intercepts the on_end event of a span to evaluate it against tail sampling rules.

The processor uses TailSamplingSpanInfo to provide context to the sampling decision function.
Sources: logfire/sampling/_tail_sampling.py:55-80, logfire/sampling/init.py:125-140

Level or Duration Strategy

The most common tail sampling strategy is level_or_duration. It ensures that:

1. Any span or log with a level at or above a specific threshold (e.g., notice, warn, error) is kept.
2. Any span that takes longer than a specified duration threshold is kept.
3. A background percentage of other spans is kept based on the head sampling rate.

Sources: logfire/sampling/init.py:52-95, tests/test_tail_sampling.py:15-30

Option	Type	Default	Description
level_threshold	LevelName	'notice'	Keeps all spans/logs at or above this level.
duration_threshold	float	1.0	Keeps all spans longer than this many seconds.
head	float	1.0	The primary sampling rate for all traces.
background_rate	float	1.0	Sampling rate for spans that don't meet level/duration criteria.

Sources: logfire/sampling/init.py:60-75, logfire/_internal/config.py:465-480

Span Levels in Sampling

Logfire introduces a SpanLevel class used specifically within tail sampling logic to allow for easy comparisons between log levels.

Comparison Logic

SpanLevel objects can be compared against string level names or numeric values. This is used by the TailSamplingSpanInfo to determine if a span has reached the required importance for retention.

# Internal comparison examples
warn = SpanLevel(13) # 'warn'
assert warn >= 'notice'  # True
assert warn < 'error'    # True

Sources: logfire/sampling/_tail_sampling.py:30-50, tests/test_tail_sampling.py:270-295

Configuration and Initialization

Sampling is typically configured during the logfire.configure() call. The configuration is then internalized into the LogfireConfig object which sets up the appropriate OpenTelemetry providers.

Validation Rules

Logfire enforces specific constraints on sampling rates to prevent invalid configurations:

• 0.0 <= background_rate <= head <= 1.0
• If background_rate is specified, head must also be specified.
• Rates cannot be negative or greater than 1.0.

Sources: logfire/sampling/init.py:95-110, tests/test_tail_sampling.py:315-325

Summary

Sampling strategies in Logfire provide a robust mechanism for balancing data density with infrastructure costs. By combining Head Sampling for overall volume control and Tail Sampling for intelligent retention of critical spans (errors and performance bottlenecks), Logfire ensures that developers have the necessary visibility into production issues without being overwhelmed by routine telemetry.
Sources: logfire/sampling/init.py:12-25, logfire/_internal/config.py:465-475