Event Context Variables

Relevant source files

The following files were used as context for generating this wiki page:

• dbt_common/events/base_types.py
• dbt_common/events/functions.py
• dbt_common/events/event_manager.py
• dbt_common/events/logger.py
• dbt_common/events/types.py
• CHANGELOG.md

Event Context Variables represent the systemic and environmental metadata associated with internal events in the dbt-common repository. These variables ensure that every fired event carries consistent identification, environmental information, and execution context (such as thread and process IDs), facilitating machine-readable event streams and human-readable logs.

This system relies on global metadata extraction, invocation tracking, and standard Python threading/processing utilities to enrich event messages before they are dispatched to loggers or callbacks.

Core Metadata Components

Every event in the system is wrapped in an EventMsg structure which includes an EventInfo object containing the context variables. These variables are automatically populated when a BaseEvent is converted into its message format for logging.

Identification and Environmental Variables

Key variables include the invocation_id, which uniquely identifies a single execution of dbt, and extra metadata derived from environment variables.

Variable	Description	Source Logic
invocation_id	A unique ID for the current dbt run.	get_invocation_id()
extra	Custom environmental metadata.	Environment variables prefixed with DBT_ENV_CUSTOM_ENV_
ts	UTC timestamp of the event occurrence.	get_json_string_utcnow()
pid	Process ID of the executing process.	os.getpid()
thread	Name of the current executing thread.	threading.current_thread().name

Sources: dbt_common/events/base_types.py:126-136, dbt_common/events/functions.py:155-163

Architecture of Context Enrichment

The following diagram illustrates how raw events are enriched with context variables before being processed by the EventManager.

The msg_from_base_event function serves as the primary constructor for the enriched message, pulling state from both the event object and the global execution context.
Sources: dbt_common/events/base_types.py:117-139

Custom Metadata Handling

Custom context can be injected into events via environment variables. This is specifically handled by the metadata_vars logic, which filters the system environment for a specific prefix.

Metadata Variable Extraction

The system scans os.environ for keys starting with DBT_ENV_CUSTOM_ENV_. These variables are cached globally after the first access to improve performance.

def get_metadata_vars() -> Dict[str, str]:
    global metadata_vars
    if metadata_vars is None:
        metadata_vars = {
            k[len(_METADATA_ENV_PREFIX) :]: v
            for k, v in os.environ.items()
            if k.startswith(_METADATA_ENV_PREFIX)
        }
    return metadata_vars

Sources: dbt_common/events/functions.py:155-163

Logger Context Integration

Loggers utilize these context variables differently based on the output format. For instance, the _TextLogger uses the thread name and ts (timestamp) for console formatting, while the _JsonLogger serializes the entire EventInfo dictionary.

Sequence of Contextual Logging

Historical Changes

Significant updates to context variables include:

• Version 1.15.0: Added the invocation_started_at global field for tracking execution timing.
• Version 1.16.0: Introduced a sequence number to records and a new invocation context accessor.
• Version 1.27.0: Began tracking adapters within the invocation context.

Sources: CHANGELOG.md:167, CHANGELOG.md:180, CHANGELOG.md:255

Summary

Event Context Variables provide the necessary telemetry for dbt operations. By decoupling the event data (the "what") from the execution context (the "where" and "when"), the system ensures that logs are high-fidelity and actionable. The BaseEvent and EventManager work in tandem to ensure that variables like pid, thread, and invocation_id are consistently applied across all event types, from deprecations to node execution.