Unit Testing Models
Relevant source files
The following files were used as context for generating this wiki page:
Unit testing in dbt-core allows developers to validate the logic of individual models by providing mocked inputs (given) and verifying that the resulting output matches an expected dataset (expect). This framework is designed to test SQL transformations in isolation from live warehouse data, using small, hand-crafted datasets defined in YAML or CSV fixtures.
The system handles the orchestration of discovering dependencies, fetching or inferring schemas for mocked relations, and generating a single "diff" query that compares actual model output against expected results. Unit tests are integrated into the standard dbt DAG and can be selected using specific Node Selectors.
Architecture and Execution Pipeline
The unit test rendering process follows a structured three-phase pipeline to ensure all necessary metadata and schemas are available before generating the final SQL instruction.
The pipeline is designed to be asynchronous where possible, specifically during schema fetching to minimize blocking operations.
Sources: crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:88-100, crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:113-163
Phase 1: Discovery
During this phase, the system evaluates Jinja expressions for each given input to resolve the actual relations (models, sources, or seeds) being mocked. It also checks if these relations have cached schemas or if they need to be fetched from the warehouse.
Sources: crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:495-538
Phase 2: Schema Fetching and Inference
If a schema is missing from the cache, the system attempts to fetch it. For models currently under test, the schema can be inferred by creating an empty temporary relation from the compiled SQL or by using a query-schema path that probes the metadata.
Sources: crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:271-330, crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:693-764
Phase 3: SQL Rendering
The final phase constructs a WITH clause where each mocked input is represented as a Common Table Expression (CTE). The model under test is then executed against these CTEs. The system produces a query that performs a UNION ALL between the "actual" results and "expected" results, tagged with an actual_or_expected column for comparison.
Sources: crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:787-835
Component Data Structures
DbtUnitTest
The core data structure representing a unit test node in the manifest. It inherits from standard node attributes but includes specific unit test fields.
| Field | Type | Description |
|---|---|---|
model | String | The name of the model being tested. |
given | Vec<Given> | List of mocked inputs and their associated data/fixtures. |
expect | Expect | The expected output data/fixture for the test. |
overrides | Option<UnitTestOverrides> | Context overrides for macros, vars, and env_vars. |
tested_node_unique_id | Option<String> | Unique ID of the model being tested (version-aware). |
Sources: crates/dbt-parser/src/resolve/resolve_tests/resolve_unit_tests.rs:189-232, crates/dbt-docs-server/src/handlers/tests.rs:125-135
Fixture Formats
Unit tests support multiple data formats for given and expect blocks:
- CSV: External files or inline strings parsed into rows.
- Dict: Inline YAML list of mappings.
- SQL: Raw SQL expressions or fixture files injected verbatim.
Sources: crates/dbt-parser/src/resolve/resolve_tests/resolve_unit_tests.rs:155-187, crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:421-489
Logic and Implementation Details
Versioned Models
Unit tests automatically handle versioned models. If a model has multiple versions, a separate unit test node is created for each included version, appending .v{version} to the unique_id. Users can use include or exclude configuration to target specific model versions.
Sources: crates/dbt-parser/src/resolve/resolve_tests/resolve_unit_tests.rs:242-297
Context Overrides
Unit tests allow mocking the environment via the overrides block. This is critical for testing logic dependent on time or environment variables.
Literal Formatting
Mock data is converted into SQL literals compatible with the target adapter. For complex types like BigQuery STRUCT or Snowflake VARIANT, the system performs specific transformations (e.g., PARSE_JSON for Snowflake).
Sources: crates/dbt-tasks-sa/src/renderable/renderable/unit_test.rs:1042-1081
Selection and Filtering
Unit tests are first-class citizens in the node selection logic. They can be specifically targeted using the test_type method.
| Selector | Value | Result |
|---|---|---|
test_type:unit | N/A | Selects all unit tests. |
unit_test:<name> | unit_test_name | Selects a specific unit test by name or package-qualified name. |
test_type:data | N/A | Excludes unit tests, selecting only data tests. |
Sources: crates/dbt-scheduler/src/node_selector.rs:242-255, crates/dbt-scheduler/src/node_selector.rs:479-497
API and Documentation
The dbt-docs-server provides endpoints to retrieve unit test details and summaries. Unit tests are stored in the dbt.unit_tests parquet table and surfaced via the REST API.
- Detail Endpoint:
GET /api/v1/tests/:idreturns aUnitTestDetailobject containing common test attributes plus fixture data (givenandexpect). - List Endpoint:
GET /api/v1/testsreturns a paginated list where unit tests are identified byresource_type: "unit_test".
Sources: crates/dbt-docs-server/src/handlers/tests.rs:252-286, crates/dbt-docs-server/src/handlers/tests.rs:361-383
Summary
Unit testing models provides a robust mechanism for regression testing and logic validation without incurring the cost or complexity of full warehouse runs. By utilizing the 3-phase render pipeline, support for versioned models, and comprehensive context overrides, the system ensures that SQL transformations behave as expected across diverse environments and adapter types.