Tracing & Telemetry
Relevant source files
The following files were used as context for generating this wiki page:
Tracing and telemetry in the dbt-core Rust rewrite provide a structured, high-performance pipeline for capturing execution data, logs, and performance metrics. Built on top of the standard Rust tracing ecosystem, the system extends native capabilities to support dbt-specific event schemas, multi-sink exports (JSONL, Parquet, OTLP), and complex terminal formatting. It differentiates between developer-oriented diagnostic tracing and user-facing structured telemetry, ensuring that execution metadata can be consumed by both humans and automated analysis tools.
The architecture is layered, separating generic telemetry records and middleware from dbt-specific logic and formatting. The dbt-tracing crate provides the generic substrate, while dbt-common::tracing integrates these utilities with the dbt/Fusion runtime behavior, such as CLI layer assembly and process span attributes. Public event schemas are managed in dbt-telemetry, which defines the canonical shape for structured events exported as Parquet or OTLP.
Sources: crates/dbt-tracing/README.md:1-20, crates/dbt-common/src/tracing/README.md:1-15
Architecture Overview
The system operates as a bridge between application code and multiple output sinks. It intercepts native tracing spans and events, materializes them into structured telemetry records, and processes them through a pipeline of middlewares and consumer layers.
The TelemetryDataLayer is the single native tracing_subscriber::Layer that acts as the entry point. It handles the correlation of records with trace IDs extracted from the root context (typically an Invocation) and manages per-consumer span filtering decisions.
Sources: crates/dbt-tracing/README.md:22-55, crates/dbt-common/src/tracing/README.md:18-35
Core Components
TelemetryDataLayer
This is the bridge between native tracing and the structured pipeline. It converts unstructured calls into SpanStartInfo, SpanEndInfo, and LogRecordInfo. It uses TelemetryDataLayerConfig callbacks to provide fallback attributes for unstructured tracing calls, ensuring that even standard logs are captured within the system.
Sources: crates/dbt-tracing/README.md:104-115
Middleware Pipeline
Middlewares can transform or drop telemetry data before it reaches any consumers. Order is significant; for instance, filtering repeated parsing errors occurs before metrics aggregation to ensure counters reflect the final filtered state.
- TelemetryParsingErrorFilter: Filters repeated errors.
- TelemetryMetricAggregator: Runs last to see the final severity/outcome state for invocation metrics.
Sources: crates/dbt-common/src/tracing/README.md:46-55, crates/dbt-tracing/README.md:117-151
Consumer Layers
Consumers are read-only sinks for processed records. They determine which events they receive using TelemetryOutputFlags.
| Output Flag | Description |
|---|---|
EXPORT_JSONL | Machine-readable JSONL writers. |
EXPORT_PARQUET | Arrow/Parquet writers (requires Arrow serialization). |
EXPORT_OTLP | OpenTelemetry Protocol exporters. |
OUTPUT_CONSOLE | Human-readable terminal output. |
OUTPUT_LOG_FILE | Human-readable unstructured log files (dbt.log). |
| Sources: crates/dbt-tracing/README.md:270-290 |
Data Models and Event Schemas
The dbt-telemetry crate owns the public protobuf event types and trait implementations. These schemas are the source of truth for Fusion structured events and must remain backwards-compatible.
Event Types and Attributes
Events are categorized into spans (representing duration) and logs (discrete points in time).
- Spans:
Invocation,PhaseExecuted,NodeProcessed,HookProcessed. - Logs:
LogMessage,UserLogMessage(from Jinja),ListItemOutput.
Node Information Structure
When nodes (models, tests, seeds, etc.) are processed, specific metadata is captured to enable interactive lineage and execution info in the docs server.
| Field | Source/Logic |
|---|---|
unique_id | Unique dbt node identifier. |
node_status | Formatted status (pass, warn, error, skip). |
materialization | Strategy used (table, view, incremental, etc.). |
node_relation | Database, schema, and identifier components. |
execution_time | Duration captured from span start to end. |
Sources: crates/dbt-telemetry/README.md:1-20, crates/dbt-common/src/tracing/layers/json_compat_layer.rs:90-145
Formatting and User Presentation
User-facing CLI and file rendering is managed through specialized formatters. These handle logic for colorization, fixed-width alignment, and dbt-specific presentation conventions like the "Started" and "Finished" lines for models.
Node Outcome Formatting
The system maps internal NodeOutcome and NodeSkipReason enums to human-readable strings.
| Internal Outcome | Skip Reason | Display Status | Color |
|---|---|---|---|
Success | N/A | pass / success | Green |
Error | N/A | error | Red |
Skipped | Upstream | skipped | Yellow |
Skipped | Cached | reused | Green |
Sources: crates/dbt-common/src/tracing/formatters/node.rs:180-215
JSON Compatibility Layer
To maintain backward compatibility with legacy dbt-core JSON logs, the JsonCompatLayer translates modern structured events back into the specific JSON format expected by older ecosystem tools. It handles version reporting, argument logging, and specific event codes (e.g., Q039 for CommandCompleted).
Sources: crates/dbt-common/src/tracing/layers/json_compat_layer.rs:150-180, crates/dbt-common/src/tracing/layers/json_compat_layer.rs:325-360
Implementation and Integration
Downstream developers emit structured attributes using re-exported helpers. This ensures that callsite locations and TelemetryContext are automatically injected.
use dbt_common::{create_info_span, create_root_info_span};
use dbt_telemetry::{Invocation, PhaseExecuted};
// Create a root span for a dbt invocation
let root = create_root_info_span(Invocation {
invocation_id: invocation_id.to_string(),
..Default::default()
});
let _root_guard = root.enter();
// Create a child span for a specific execution phase
let phase = create_info_span(PhaseExecuted::start_general(phase)).entered();
Sources: crates/dbt-common/src/tracing/README.md:73-85, crates/dbt-tracing/README.md:215-235
Graceful Shutdown
Because many sinks (like Parquet and background JSONL writers) are buffered for performance, the system provides a TelemetryHandle for graceful shutdown. This ensures that all buffered records are flushed to disk or exported to remote endpoints before the process exits.
Sources: crates/dbt-common/src/tracing/README.md:41-45
Summary
The Tracing & Telemetry system provides a robust infrastructure for observability within dbt-core. By leveraging a middleware-based pipeline and distinct consumer layers, it achieves high performance while supporting diverse output needs—from real-time TUI progress updates to long-term performance analysis via Parquet artifacts. The separation of generic tracing primitives in dbt-tracing from dbt-specific schemas in dbt-telemetry ensures the system remains extensible as the project evolves.