Dataclass Schema & Serialization

Relevant source files

The following files were used as context for generating this wiki page:

• dbt_common/dataclass_schema.py
• dbt_common/contracts/config/base.py
• dbt_common/contracts/metadata.py
• dbt_common/semver.py
• dbt_common/exceptions/base.py
• tests/unit/test_model_config.py

The Dataclass Schema & Serialization module provides a robust framework for defining, validating, and converting data structures within the dbt ecosystem. It leverages the mashumaro library to extend Python's standard dataclasses with features such as JSON schema generation, automated serialization/deserialization, and complex merging behaviors.

This system ensures that internal contracts (such as configurations, metadata, and versioning) remain consistent across different modules and adapter implementations. It integrates with jsonschema for rigorous validation and provides specific mixins to handle extensible properties and specialized data types like semantic versions and dates.

Core Mixins and Configuration

The primary interface for schema-aware dataclasses is the dbtClassMixin. This mixin pulls in serialization methods (to_dict, from_dict) and adds validation capabilities.

Mashumaro Integration

The module defines dbtMashConfig to control the serialization engine. Key configurations include:

• Lazy Compilation: Improves startup performance by delaying the generation of serialization functions.
• Omit None: Option to exclude None values from the resulting dictionary.
• Serialization Strategy: Custom handlers for datetime and bool types to ensure backward compatibility and specific string formats.

Sources: dbt_common/dataclass_schema.py:44-77, pyproject.toml:98-100

Mixin Hierarchy

The following diagram illustrates the relationship between the core mixins and external libraries:

The hierarchy ensures that all dbt contracts inherit basic serialization and specialized validation.
Sources: dbt_common/dataclass_schema.py:79-110, dbt_common/dataclass_schema.py:165-171

Validation and JSON Schema

The module uses jsonschema (specifically Draft 7) to validate data against the structure defined in the dataclasses.

Schema Generation

The json_schema() method uses mashumaro.jsonschema.build_json_schema to generate a JSON representation of the class. By default, additionalProperties is set to False to enforce strict schema adherence, unless the ExtensibleDbtClassMixin is used.

Validation Flow

When validate() is called, the following sequence occurs:

1. The class generates its own JSON schema.
2. A Draft7Validator is instantiated with this schema.
3. The input data is checked for errors.
4. If errors exist, a ValidationError (subclassed from jsonschema.ValidationError) is raised.

Sources: dbt_common/dataclass_schema.py:27-28, dbt_common/dataclass_schema.py:112-124

Configuration Merging Behaviors

A critical feature of the serialization system is the ability to merge configuration dictionaries based on predefined behaviors. This is primarily handled in BaseConfig and the MergeBehavior enum.

Merge Behavior Types

Behavior	Logic	Metadata Key
Clobber	Default behavior; the new value overwrites the existing one.	merge: 3
Append	Combines two lists or converts single values into a concatenated list.	merge: 1
Update	Performs a dictionary update (dict.update()).	merge: 2
DictKeyAppend	Merges dictionaries where values are lists; allows the + prefix to extend lists.	merge: 4
Object	Deeply merges objects of the same type (used for complex config nested classes).	merge: 5

Sources: dbt_common/contracts/config/base.py:176-189, dbt_common/contracts/config/base.py:218-274

Data Flow: config.update_from()

The update_from method allows a configuration object to be updated with new dictionary data while respecting field-level merge metadata.

Specialized Serializers

The system includes specialized types and serialization strategies to handle data that does not have a direct 1:1 JSON mapping.

Semantic Versioning

The VersionSpecifier class (inheriting dbtClassMixin) provides logic for parsing, comparing, and serializing semantic versions based on a complex regex.

• Parsing: from_version_string uses _VERSION_REGEX to extract major, minor, patch, and prerelease components.
• Comparison: Implements __lt__, __gt__, and __eq__ to facilitate version range reduction.

Sources: dbt_common/semver.py:18-80, dbt_common/semver.py:108-140

Validated and Enumerated Types

• ValidatedStringMixin: A mixin that enforces a regex check during deserialization.
• StrEnum: A mixin for string-based Enums that ensures they serialize to their value rather than the Enum object.
• DateTimeSerialization: Ensures datetime objects are serialized to ISO-8601 format, defaulting to UTC if timezone info is missing.

Sources: dbt_common/dataclass_schema.py:31-41, dbt_common/dataclass_schema.py:147-163

Error Handling in Serialization

Serialization errors are integrated into the project's broader exception hierarchy.

Exception	Usage
ValidationError	Raised when data fails to meet the jsonschema requirements of a dataclass.
DbtRuntimeError	Provides a validator_error_message helper to format jsonschema errors into human-readable paths (e.g., at path [columns][0]: ...).
DbtInternalError	Raised during merging if types are conflicting (e.g., trying to merge a dict into a list).

Sources: dbt_common/exceptions/base.py:104-114, dbt_common/contracts/config/base.py:266-269

Summary

The Dataclass Schema & Serialization module is the foundational layer for data integrity in dbt-common. By combining mashumaro for speed and jsonschema for correctness, it provides a declarative way to handle complex configuration lifecycles—including validation, type coercion, and rule-based merging—that are essential for multi-adapter environments.