Auto-Tracing (AST Rewriting)
Relevant source files
The following files were used as context for generating this wiki page:
Auto-Tracing is a sophisticated mechanism in Pydantic Logfire that enables automatic instrumentation of Python code without requiring manual manual span creation for every function. It achieves this by intercepting the Python import process and rewriting the Abstract Syntax Tree (AST) of targeted modules to inject telemetry logic.
The system is designed to provide "unparalleled visibility" into application behavior by automatically wrapping function bodies in Logfire spans. This process is configurable, allowing developers to filter specific modules and set performance thresholds for when spans should be recorded.
Sources: CLAUDE.md:16, README.md:11, logfire/_internal/auto_trace/import_hook.py:23-28
Architecture Overview
The Auto-Tracing system consists of three primary layers: the configuration interface, the import hook, and the AST rewriter.
The diagram shows the lifecycle of a module from the initial installation of auto-tracing to the runtime execution of instrumented code.
Sources: logfire/_internal/auto_trace/import_hook.py:23-50, tests/test_auto_trace.py:30-45
Core Components
LogfireFinder (Import Hook)
The LogfireFinder is a MetaPathFinder implementation inserted into sys.meta_path. It monitors the import system to identify modules that match the user's defined filters. When a match is found, it intercepts the standard loading process.
Key responsibilities:
- Iterating through other meta path finders to locate source code and filenames.
- Filtering modules based on the
AutoTraceModuledata structure. - Compiling the modified AST into an executable format.
Sources: logfire/_internal/auto_trace/import_hook.py:23-68
AST Rewriting Logic
The rewriting logic parses the module's source code into an AST and traverses it to find function and method definitions. It wraps the bodies of these functions in with logfire_span(): blocks.
Key features of the transformation:
- Nested Support: Correctly identifies and instruments nested functions and classes.
- Async Support: Handles
async deffunctions appropriately. - Exclusion: Respects the
@no_auto_tracedecorator to skip specific functions or classes. - Metadata Injection: Injects line numbers, file paths, and function names into the generated spans.
Sources: logfire/_internal/auto_trace/rewrite_ast.py:10-50, tests/test_auto_trace.py:126-150
LogfireLoader
A custom Loader produced by the finder. Instead of executing the original byte code, it executes the modified AST within the module's namespace via exec_module.
Sources: logfire/_internal/auto_trace/import_hook.py:92-115
Configuration and Filtering
Auto-tracing is activated via logfire.install_auto_tracing(). This function accepts various parameters to control the scope and sensitivity of the instrumentation.
| Parameter | Type | Description |
|---|---|---|
modules | list[str] | Callable | List of module names or a filter function to identify targets. |
min_duration | int | float | Minimum nanoseconds/seconds a function must take before a span is recorded (default is 0). |
check_imported_modules | Literal['ignore', 'warn', 'error'] | Strategy for handling modules that were already imported before auto-tracing was installed. |
Sources: tests/test_auto_trace.py:30-40, logfire/_internal/config.py:465-480, logfire-api/logfire_api/init.py:123
The AutoTraceModule Object
The system uses the AutoTraceModule class to represent modules during the filtering phase.
This diagram represents the data structure used to evaluate whether a module should be instrumented.
Sources: logfire/_internal/auto_trace/import_hook.py:26, tests/test_auto_trace.py:195-215
Runtime Execution Flow
When an instrumented function is called, the injected code interacts with the Logfire instance to manage spans.
This sequence illustrates how the injected with block ensures that spans are tracked and errors are captured even though the developer wrote no tracing code.
Sources: tests/test_auto_trace.py:75-120, logfire/_internal/auto_trace/rewrite_ast.py:100-110
Implementation Details
Minimum Duration Threshold
The min_duration setting allows for "sampling" based on performance. If a function's execution time is below this threshold, the system may suppress the span to reduce noise. During the first call, durations are measured; subsequent calls create spans if the threshold was exceeded.
Sources: tests/test_auto_trace.py:230-260
Handling Already Imported Modules
Because AST rewriting happens during the import phase, modules imported before install_auto_tracing() is called cannot be instrumented. Logfire provides checks to warn or raise exceptions if the user attempts to auto-trace a module that is already resident in sys.modules.
Sources: tests/test_auto_trace.py:100-115, logfire/_internal/auto_trace/init.py
Performance Considerations
- Compile-time Overhead: Parsing and rewriting the AST adds a small delay to the initial import of a module.
- Runtime Overhead: Each function call now includes a context manager entry and exit, though this is minimized by the efficient internal span management of Logfire.
Sources: logfire/_internal/auto_trace/import_hook.py:60-65, logfire/_internal/config.py:650-660
Summary
Auto-Tracing (AST Rewriting) is a key feature of the Pydantic Logfire SDK that provides a "zero-effort" entry point for deep application observability. By leveraging Python's sys.meta_path and ast modules, it transparently transforms standard code into instrumented code, providing comprehensive function-level telemetry while maintaining developer productivity.