Exceptions Framework

Relevant source files

The following files were used as context for generating this wiki page:

• dbt_common/exceptions/base.py
• dbt_common/exceptions/jinja.py
• dbt_common/clients/jinja.py
• dbt_common/helper_types.py
• dbt_common/dataclass_schema.py
• dbt_common/semver.py

The Exceptions Framework in dbt-common provides a structured, hierarchical approach to error handling across dbt core and its adapters. It establishes a base set of exception classes that support features such as secret scrubbing, node-context tracking, and stack trace processing to ensure that error messages are both secure and actionable for developers.

The framework is designed to handle various failure modes, including Jinja compilation errors, database execution failures, configuration mismatches, and validation errors. By providing a unified DbtBaseException, the system ensures consistent JSON serialization and error reporting across different modules.
Sources: dbt_common/exceptions/base.py:22-31, dbt_common/exceptions/jinja.py:8-10

Core Architecture and Base Classes

The hierarchy is rooted in DbtBaseException, which defines the standard interface for all dbt-specific errors. It includes a data() method that returns a JSON-serializable dictionary containing the exception type and message.

Primary Exception Hierarchy

The following diagram illustrates the relationship between the primary exception classes:

The framework distinguishes between internal errors (bugs in dbt) and runtime errors (issues with user code or environment).
Sources: dbt_common/exceptions/base.py:22-80

Key Base Components

Class	Description	Key Attributes
DbtBaseException	The root exception for the framework.	CODE, MESSAGE
DbtInternalError	Used for unexpected internal state or logic failures.	stack, msg (scrubbed)
DbtRuntimeError	Base for errors occurring during dbt execution or compilation.	node, stack, msg
DbtConfigError	Raised for errors in dbt_project.yml or profile configurations.	project, path, result_type

Sources: dbt_common/exceptions/base.py:22-140

Security and Privacy: Secret Scrubbing

A critical feature of the exceptions framework is the automatic scrubbing of sensitive information from error messages. This prevents environment variables marked as "secrets" from being leaked in logs or console output.

The scrub_secrets function identifies strings matching values found in environment variables prefixed with DBT_ENV_SECRET_ and replaces them with *****. This logic is automatically applied in the constructors of DbtInternalError and DbtRuntimeError.
Sources: dbt_common/exceptions/base.py:10-20, dbt_common/exceptions/base.py:36-37, dbt_common/exceptions/base.py:73-74

def scrub_secrets(msg: Any, secrets: List[str]) -> Any:
    scrubbed = str(msg)
    for secret in secrets:
        scrubbed = scrubbed.replace(secret, "*****")
    return msg if str(msg) == scrubbed else scrubbed

Sources: dbt_common/exceptions/base.py:14-20

Jinja Compilation Exceptions

The framework provides specialized exceptions for failures during the parsing and rendering of Jinja templates. These errors often include position tracking to pinpoint where a block was incorrectly defined or where a tag was left unclosed.

Specialized Jinja Errors

Exception Class	Trigger Condition
BlockDefinitionNotAtTopError	A dbt block (like macro or snapshot) is defined inside a control flow (like if or for).
MissingCloseTagError	The end of the file was reached without finding a matching {% end... %} tag.
NestedTagsError	A new block started before the previous one was closed.
UnexpectedControlFlowEndTagError	An end tag like {% endif %} was found without a preceding if.

Sources: dbt_common/exceptions/jinja.py:11-73

Template Error Handling Flow

The catch_jinja context manager is used to translate standard Jinja2 exceptions into the dbt Exceptions Framework.

Context-Aware Errors

DbtRuntimeError and its subclasses support a "node" attribute. This allows the framework to associate an error with a specific dbt resource (e.g., a model, seed, or macro).

The node_to_string method attempts to build a descriptive identifier for the failing node using attributes like resource_type, name, and original_file_path. If the node has been compiled, the data() method will include the raw_code and compiled_code in its output.
Sources: dbt_common/exceptions/base.py:84-106

Warning and Error Options Configuration

The framework includes a mechanism to promote warnings to errors or silence specific events via WarnErrorOptionsV2. This class manages lists of event names and provides logic to determine if an event should trigger a failure.

Evaluation Logic for Warnings/Errors

Database and Command Errors

The framework extends DbtRuntimeError to handle system-level and database-specific failures:

1. DbtDatabaseError: Adds compiled code paths to the stack trace if available.
2. UnexpectedNullError: A specialized database error for when a non-null field returns a null value during a query.
3. CommandError: Used for external subprocess failures, ensuring that the command arguments are also scrubbed for secrets before being displayed.
   Sources: dbt_common/exceptions/base.py:155-189

Summary

The Exceptions Framework in dbt-common acts as the backbone for error reporting. By integrating secret scrubbing, node-level context, and specialized Jinja handling, it ensures that errors are safe to log while providing the necessary technical detail for debugging. The framework's hierarchical nature allows developers to catch broad categories of errors (like DbtRuntimeError) or specific failures (like MissingCloseTagError) as needed.