Testing Tools & Mocks
Relevant source files
The following files were used as context for generating this wiki page:
The dbt-common repository provides a suite of testing utilities, environment configurations, and mocking mechanisms designed to ensure the reliability of shared components used across dbt-core and various adapter implementations. These tools include automated unit testing frameworks, specialized event catching for telemetry verification, and Jinja template debugging utilities.
The testing architecture relies on hatch for environment management and pytest as the primary test runner, supplemented by specialized classes for capturing system events and simulating complex Jinja environments.
Sources: CONTRIBUTING.md:5, pyproject.toml:58-63
Test Environment Management
The project utilizes hatch to manage isolated Python environments for testing, linting, and building. This ensures that tests are run in a consistent environment regardless of the host system's configuration.
Hatch Environments
The pyproject.toml defines specific environments with their respective dependencies:
| Environment | Features | Purpose |
|---|---|---|
default | lint, test | Standard development environment. |
test | test | Optimized for running unit tests. |
lint | lint | Static analysis and style checking. |
Sources: pyproject.toml:74-95
Testing Commands
Developers interact with the testing suite through standardized hatch run commands:
# Run all unit tests with coverage reporting
hatch run test:unit
# Run static analysis (Black, Flake8, Mypy)
hatch run lint:all
Sources: CONTRIBUTING.md:105-115, pyproject.toml:84-85
Event Catching and Verification
The EventCatcher utility is a critical tool for verifying that the system correctly fires telemetry and logging events. It acts as a callback listener that intercepts events fired via the EventManager.
Event Management Flow
The following diagram illustrates how the EventCatcher integrates with the event system during a test execution:
This flow shows the registration of the catcher as a callback and the subsequent collection of events for validation.
Sources: tests/unit/test_functions.py:32-38, CHANGELOG.md:139-141
Event Validation Utilities
Tests use the EventCatcher to assert properties of fired events, such as their log level or message content.
ctx_set_event_manager: Used to inject a specific manager instance into the current thread context for testing.caught_events: A list within theEventCatcherthat stores every event intercepted during the test run.
Sources: tests/unit/test_functions.py:32-38, tests/unit/test_functions.py:86-88
Jinja Template Mocking and Debugging
The Jinja client includes specialized environments and markers to facilitate testing and debugging of template rendering logic.
Template Debugging with Linecache
A unique feature of the MacroFuzzEnvironment is the ability to inject rendered source code into the Python linecache. This allows developers to use standard Python debugging tools to step through rendered Jinja templates as if they were source files on disk.
def _linecache_inject(source: str, write: bool) -> str:
# Stashes rendered source in linecache for debugging
# filename is generated or assigned based on random bytes
cache_entry = (len(source), None, [line + "\n" for line in source.splitlines()], filename)
linecache.cache[filename] = cache_entry
return filename
Sources: dbt_common/clients/jinja.py:73-95
Native Type Markers
For testing "Native Jinja" (where templates return Python types instead of strings), the system provides several markers used to verify and control type evaluation:
TextMarker: Prevents numeric strings from being evaluated as numbers.BoolMarker/NumberMarker: Forces validation that the rendered result matches the expected Python primitive.
Sources: dbt_common/clients/jinja.py:180-205
Record and Replay Mechanism
The project implements a record/replay framework that allows developers to capture the inputs and outputs of function calls during a "live" run and replay them during tests. This acts as a sophisticated mocking layer for external calls and complex internal logic.
Components of Record/Replay
| Component | Description |
|---|---|
Recorder | Captures function calls and serializes them to a recordings.json file. |
@auto_record_function | Decorator used to automatically mark functions for recording without manual boilerplate. |
RecordReplayIssue | A specialized event fired when discrepancies or issues occur during the record/replay cycle. |
Sources: CHANGELOG.md:58-60, CHANGELOG.md:180-185, dbt_common/events/types.py:157-164
Record/Replay Flow
The sequence diagram demonstrates the transition from capturing data to utilizing it as a mock in subsequent test runs.
Sources: CHANGELOG.md:175-185, CHANGELOG.md:210-212
System Client Mocks
Tests for the system client (file I/O, shell commands) avoid global mocks of the os or shutil modules to ensure cross-platform compatibility (Windows vs. Linux). Instead, they utilize temporary directories and manual cleanup.
File System Testing
The SystemClient unit tests create real files and directories within a tempfile.mkdtemp() scope to verify actual OS behavior.
def setUp(self) -> None:
self.tmp_dir = mkdtemp()
self.profiles_path = "{}/profiles.yml".format(self.tmp_dir)
def tearDown(self) -> None:
shutil.rmtree(self.tmp_dir)
Sources: tests/unit/test_system_client.py:20-38
Shell Command Verification
The TestRunCmd class tests the run_cmd utility by executing native echo commands (e.g., cmd /C echo on Windows vs echo on Linux) to verify that exit codes and standard output streams are handled correctly across different operating systems.
Sources: tests/unit/test_system_client.py:75-84
Summary
Testing in dbt-common is characterized by a "test-what-you-run" philosophy, using isolated environments and real system calls where possible, while providing highly specialized capturing tools for events and Jinja rendering. The introduction of the record/replay mechanism significantly reduces the need for manual mock objects for complex data flows.