Developing a Custom Adapter
Relevant source files
The following files were used as context for generating this wiki page:
Developing a custom adapter for dbt allows the dbt-core framework to interact with various storage and compute platforms. This process involves implementing specific Python protocols and SQL macros that translate dbt commands into platform-specific actions. The project follows a monorepo structure where dbt-adapters provides the base framework and protocols, while individual adapters like dbt-postgres or dbt-snowflake extend this functionality.
The architecture is divided into two major types: BaseAdapter, which defines mandatory functionality for any dbt integration, and SQLAdapter, which provides a specialized foundation for SQL-based platforms. Custom development typically focuses on four key components: connection management, dialect translation (macros), relation caching, and integration metadata.
Sources: dbt-adapters/README.md, CLAUDE.md:68-80
Adapter Architecture
The hierarchy of dbt adapters is designed for maximum code reuse. Most SQL-based adapters inherit from SQLAdapter, which in turn inherits from BaseAdapter.
The diagram above illustrates the inheritance structure within the monorepo, showing how specialized adapters like Redshift leverage the PostgreSQL foundation.
Sources: CLAUDE.md:14-25, dbt-adapters/README.md:13-21
Core Components
A custom adapter is composed of several critical subsystems:
| Component | Responsibility | Relevant Files |
|---|---|---|
| Connections | Managing thread-safe connection pooling and compute credentials. | connections.py |
| Dialect | Translating dbt requests into platform-specific SQL via macros. | relation.py, column.py, macros/ |
| Caching | Local management of relation metadata to optimize performance. | cache.py |
| Integration | Interface between dbt-core and the concrete implementation. | impl.py, plugin.py, capability.py |
Sources: dbt-adapters/README.md:27-68, CLAUDE.md:83-95
Development Workflow
Development occurs within the specific adapter's directory in the monorepo. The workflow utilizes hatch for environment management and changie for versioning.
This flowchart describes the standard path for adding features or fixing bugs in a specific adapter.
Sources: AGENTS.md:23-31, CLAUDE.md:31-48, CONTRIBUTING.md:43-52
Implementation Steps
- Choose Base Class: Select
SQLAdapterfor standard SQL databases orBaseAdapterfor non-SQL or highly custom platforms. - Define Credentials: Create a
Credentialsdataclass to hold connection parameters (e.g., host, port, user). - Implement Required Methods: At minimum, implement
ConnectionManager.exception_handler(),.open(), and.cancel(). - Register Plugin: Use the
AdapterPluginclass in the adapter's__init__.pyto register the adapter with dbt's factory. - Macro Overrides: Implement adapter-specific SQL in the
include/{adapter}/macros/directory, prefixing macro names with{adapter}__.
Sources: CLAUDE.md:295-310, AGENTS.md:73-82
Testing with dbt-tests-adapter
All adapters must pass a standardized suite of tests provided by the dbt-tests-adapter package. These tests ensure consistent behavior across different data platforms.
Test Categories
- Basic Tests: Mandatory tests covering simple materializations, snapshots, incremental models, and documentation generation.
- Optional Tests: Second-order functionality such as constraints, grants, and materialized views.
The sequence diagram shows how the shared test suite interacts with the custom adapter implementation to verify database behavior.
Sources: dbt-tests-adapter/README.md:12-28, CLAUDE.md:105-125
Advanced Features
Capabilities Declaration
Adapters must explicitly declare their supported features using the CapabilityDict. This allows dbt-core to optimize execution based on what the platform can handle.
Sources: AGENTS.md:92-100, CLAUDE.md:313
External Catalog Integrations
The framework supports external table formats like Apache Iceberg through a plugin system in dbt-adapters/src/dbt/adapters/catalogs/. Developers can implement CatalogIntegration to handle metadata for these formats.
Sources: CLAUDE.md:255-265, AGENTS.md:110-116
Event Logging
The dbt.adapters.events.types module defines the standard events that an adapter should emit during its lifecycle, such as NewConnection, SQLQuery, and CacheMiss.
Sources: dbt-adapters/src/dbt/adapters/events/types.py:75-135
Conclusion
Developing a custom adapter is a structured process of extending the BaseAdapter or SQLAdapter classes and providing a comprehensive set of SQL macro overrides. By adhering to the monorepo's established testing patterns and capability declarations, developers ensure their adapter is reliable, performant, and compatible with the broader dbt ecosystem.
Sources: README.md:20-30, CLAUDE.md:315-325