Monorepo Structure & Package Management
Relevant source files
The following files were used as context for generating this wiki page:
The dbt-adapters repository is organized as a monorepo containing the v1 generation of dbt adapters and their shared framework. This structure allows multiple independent Python packages to reside in a single repository, facilitating synchronized development and testing of related changes across the base framework and specific database adapters.
The repository centralizes the core dbt-adapters framework, the dbt-tests-adapter shared test suite, and several first-party adapters (Postgres, Redshift, Snowflake, BigQuery, Spark, and Athena). This architecture ensures that changes to the base framework can be immediately validated against all downstream adapter implementations.
Sources: README.md:12-25, CONTRIBUTING.md:23-32, CLAUDE.md:3-13
Repository Layout and Dependency Chain
The monorepo is organized into distinct subdirectories, each functioning as an independent Python package with its own configuration files (pyproject.toml, hatch.toml) and test suites.
Directory Structure
- dbt-adapters/: The base framework containing
BaseAdapter,SQLAdapter, contracts, and catalog protocols. - dbt-tests-adapter/: A shared integration test suite utilized by all adapters to ensure protocol compliance.
- dbt-{adapter}/: Specific implementations for various data platforms.
Sources: AGENTS.md:5-15, CLAUDE.md:3-13
Logical Dependency Flow
The project follows a hierarchical dependency structure where the base framework supports all adapters, and certain adapters serve as foundations for others.
The dbt-postgres package acts as a foundation for dbt-redshift, and dbt-tests-adapter is a development dependency for every adapter package.
Sources: AGENTS.md:155-160, CLAUDE.md:15-26, README.md:20-25
Environment Management with Hatch
The project utilizes hatch as the primary tool for environment management, building, and orchestration. All development commands are executed within the specific package directories rather than the repository root.
Core Hatch Commands
The following table summarizes the standard development workflow within any adapter directory:
| Command | Purpose |
|---|---|
hatch run setup | Installs the package and all repo-local dependencies in editable mode; registers git hooks. |
hatch run code-quality | Runs formatting and linting tools (Black, Flake8, MyPy, etc.). |
hatch run unit-tests | Executes Python logic tests that do not require a live database. |
hatch run integration-tests | Executes functional tests against a live database instance. |
hatch shell | Activates the virtual environment for the current package. |
| Sources: CONTRIBUTING.md:36-68, AGENTS.md:21-41, CLAUDE.md:33-45 |
Editable Installations
A critical feature of this monorepo is the use of editable installations for local packages. When hatch run setup is executed in an adapter directory (e.g., dbt-postgres), it installs the local version of dbt-adapters in editable mode. This ensures that modifications made to the base framework are immediately reflected in the adapter's environment without requiring a re-install.
Sources: CONTRIBUTING.md:40-44, CONTRIBUTING.md:83-88
Package Configuration and Versioning
Each package in the monorepo maintains its own metadata and lifecycle configuration.
Configuration Files
- hatch.toml: Defines environment-specific dependencies (e.g.,
default,ci,cd), environment variables, and scripts for localized tasks. - pyproject.toml: Contains standard Python package metadata, entry points for dbt plugins, and build system requirements.
Sources: dbt-adapters/hatch.toml:1-30, AGENTS.md:18-19, CLAUDE.md:94-106
Release and Changelog Management
The repository uses changie to manage version history across the monorepo. Instead of editing a central CHANGELOG.md directly, contributors generate small YAML files for each change.
Each adapter maintains its own release notes, which are collated into individual subdirectories during the release process.
Sources: CONTRIBUTING.md:124-140, CHANGELOG.md:1-12, AGENTS.md:148-153
Cross-Package Development Workflow
When implementing features that span multiple packages, developers must manage the local environment to ensure all components are aligned.
- Framework & Adapter Changes: Changes in
dbt-adaptersare automatically picked up by adapters via the editable installs created duringhatch run setup. - External Dependencies: If a change requires a specific branch of an external package like
dbt-common, developers must manually install that package in editable mode within the Hatch environment:hatch run pip install -e ../../dbt-common - CI Synchronization: To test cross-repo changes in CI, the
hatch.tomlof the adapter must be temporarily updated to point the dependency at a specific git branch.
Sources: CONTRIBUTING.md:83-110, CLAUDE.md:290-300
Conclusion
The monorepo structure of dbt-adapters provides a robust framework for managing the complex interplay between dbt's base adapter logic and platform-specific implementations. By leveraging hatch for localized package management and changie for distributed changelogging, the project maintains a high degree of developer velocity and code quality across its diverse set of packages.