Exception Handling & Scrubbing
Relevant source files
The following files were used as context for generating this wiki page:
Exception Handling and Scrubbing in Pydantic Logfire are critical systems designed to ensure operational reliability and data privacy. The exception handling framework provides a structured set of custom errors used to signal configuration issues or internal failures. Scrubbing is a data protection mechanism that automatically redacts sensitive information—such as credentials, keys, and personal identifiers—from logs and spans before they are exported to the backend.
These systems work together within the SDK's internal pipeline. Scrubbing is typically implemented via processor wrappers that intercept OpenTelemetry spans and log records, applying regex patterns and custom callbacks to clean attributes and messages. Exception handling ensures that these processes, as well as general SDK configuration, fail gracefully with descriptive error types.
Data Scrubbing Architecture
Scrubbing is implemented as a pre-export transformation layer. The core logic resides in the Scrubber class, which manages a set of regex patterns and a callback function to identify and redact sensitive values.
The Scrubbing Pipeline
The scrubbing process is integrated into the OpenTelemetry pipeline through MainSpanProcessorWrapper and MainLogProcessorWrapper. These wrappers intercept telemetry data before it reaches the exporters.
The diagram above shows the flow of telemetry data through the scrubbing layer before reaching the final exporter.
Sources: logfire/_internal/exporters/processor_wrapper.py:17-30, logfire/_internal/exporters/logs.py:19-32
Key Scrubbing Components
| Component | Description |
|---|---|
Scrubber | Primary class responsible for redacting values based on patterns and callbacks. |
ScrubbingOptions | Configuration dataclass for user-defined patterns and callbacks. |
ScrubMatch | Object passed to callbacks containing the path, value, and matched pattern. |
NOOP_SCRUBBER | A no-op implementation used when scrubbing is disabled. |
Sources: logfire/_internal/scrubbing.py:44-59, logfire/_internal/config.py:273-286
Default Redaction Patterns
Logfire includes a comprehensive set of default regex patterns to catch common sensitive data types, including:
- Credentials: Authorization headers, passwords, and secret keys.
- Authentication: API keys for services like OpenAI, Anthropic, and AWS.
- Identifiers: Credit card numbers and social security numbers.
- Database: Connection strings containing sensitive URIs.
Sources: logfire/_internal/scrubbing.py:108-158
Exception Handling Framework
The project uses a hierarchy of custom exceptions to manage various failure states, primarily focused on configuration and initialization.
Exception Hierarchy
This diagram illustrates the relationship between standard Python exceptions and Logfire's specialized error types.
Sources: logfire/exceptions.py:1-13, logfire/_internal/config.py:1296-1297
Core Exception Types
LogfireError: The base class for all exceptions raised by the Logfire SDK.LogfireConfigError: Raised when the SDK is provided with invalid configuration parameters, missing credentials, or unreachable API endpoints during setup.LogfireNotConfiguredWarning: A warning issued when Logfire APIs are called beforelogfire.configure()has been executed, unless suppressed by configuration.
Sources: logfire/exceptions.py:1-13, logfire/_internal/config.py:996-1002
Stack and Source Information
Logfire handles stack information to provide context for errors and spans, ensuring that logs point to user code rather than internal SDK implementation.
User Stack Level Detection
The system uses warn_at_user_stacklevel to issue warnings that appear to originate from the caller's code. It achieves this by inspecting the call stack and finding the first frame that does not belong to the logfire package.
def warn_at_user_stacklevel(message: str, category: type[Warning]):
"""Issue a warning at the first stack level outside of logfire."""
# Logic to find the stack level of user code
stacklevel = find_user_stacklevel()
warnings.warn(message, category, stacklevel=stacklevel)
Sources: logfire/_internal/stack_info.py:18-47
JSON Encoding and Serialization
To handle complex objects (including exceptions and custom types) in span attributes, the SDK employs a specialized JSON encoder.
The JSON Encoding Flow
The DataclassJSONEncoder handles the serialization of various Python structures into formats compatible with the backend.
The diagram shows how the internal encoder transforms complex Python types into JSON-serializable formats.
Sources: logfire/_internal/json_encoder.py:11-40
Configuration Options
Configuration for these systems is managed through the configure() function and the LogfireConfig class.
| Option | Type | Default | Description |
|---|---|---|---|
scrubbing | ScrubbingOptions | False | ScrubbingOptions() | Configures or disables data redacting. |
advanced.exception_callback | ExceptionCallback | None | Optional callback triggered when an exception is recorded on a span. |
ignore_no_config | bool | False | Suppresses LogfireNotConfiguredWarning. |
Sources: logfire/_internal/config.py:339-380, logfire/_internal/config.py:180-192
Summary
The Exception Handling and Scrubbing systems are foundational to Logfire's goal of providing safe and reliable observability. Scrubbing acts as a security gatekeeper, automatically cleaning telemetry data using a combination of robust regex patterns and flexible callbacks. The exception framework provides clear, actionable feedback for developers, while internal utilities for stack inspection and JSON encoding ensure that telemetry remains technically accurate and focused on relevant user code.