Data Contracts
Relevant source files
The following files were used as context for generating this wiki page:
- docs-website/src/learn/data-mesh.md
- datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/DataQualityContractSummary.tsx
- datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/FreshnessContractSummary.tsx
- datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/SchemaContractSummary.tsx
- docs-website/static/llms.txt
Data Contracts in DataHub serve as formal agreements between data producers and consumers to ensure the reliability, consistency, and quality of data products. They provide a mechanism to define expectations around schema, freshness, and data quality metrics, which are then monitored through automated Assertions.
Within a Data Mesh architecture, data contracts are critical for establishing clear agreements between decentralized domains. They allow teams to manage data products with autonomy while maintaining a high standard of interoperability and reliability across the organization.
Sources: docs-website/src/learn/data-mesh.md:124-129, docs-website/static/llms.txt:59-59
Contract Components and Structure
Data Contracts are composed of several specialized sub-contracts that target different aspects of a dataset's metadata. Each contract type maps to specific validation logic and UI components.
Contract Types
| Contract Type | Description | Key Metadata Fields |
|---|---|---|
| Schema Contract | Defines the expected structure of the data, including field paths and types. | fieldPath, type |
| Freshness Contract | Sets expectations for how often data should be updated. | schedule |
| Data Quality Contract | Defines assertions for data values, such as volume, column validity, or custom SQL checks. | scope, operator, parameters, logic |
Sources: datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/SchemaContractSummary.tsx:57-67, datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/FreshnessContractSummary.tsx:58-61, datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/DataQualityContractSummary.tsx:50-101
Architecture and Data Flow
The Data Contract system operates by linking contract definitions to DataHub's assertion framework. When a contract is defined, it utilizes one or more assertions to validate the state of the data.
Validation Logic Flow
The following diagram illustrates how the UI components interact with contract definitions and assertions to display the status of a Data Contract.
The UI components retrieve assertion information (type, scope, and parameters) and the latest result status to determine if a contract is passing, failing, or in an error state.
Sources: datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/DataQualityContractSummary.tsx:115-121, datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/FreshnessContractSummary.tsx:82-88
Implementation Details
Schema Validation
Schema contracts validate individual fields within a dataset. The system renders these as a table comparing the contract's required fields and types against the actual schema metadata.
Sources: datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/SchemaContractSummary.tsx:57-67
Freshness and SLA Monitoring
Freshness contracts specifically target the freshnessAssertion type. They summarize the update frequency schedule defined in the assertion to provide an SLA (Service Level Agreement) summary to the user.
Sources: datahub-web-react/src/app/entityV2/shared/tabs/Dataset/Validations/contract/FreshnessContractSummary.tsx:61-75
Data Quality Assertions
Data quality contracts support multiple assertion types, which are rendered dynamically based on the assertion's info:
- Dataset Assertions: Validate aggregations and scopes.
- Volume Assertions: Monitor row counts or data size.
- Field Assertions: Validate values at the column level.
- SQL Assertions: Run custom SQL logic to verify complex business rules.
- Custom Assertions: Support structured descriptions or fallback types.
Conclusion
Data Contracts are a foundational feature for implementing reliable data systems and Data Mesh architectures. By codifying expectations into schema, freshness, and quality contracts, DataHub enables automated monitoring and clear communication of data health between producers and consumers.
Sources: docs-website/src/learn/data-mesh.md:124-129, docs-website/static/llms.txt:59-59