Data Schema Models
Relevant source files
The following files were used as context for generating this wiki page:
The Data Schema Models within the flext-dbt-ldap project provide a structured framework for defining, validating, and transforming LDAP directory data into a format suitable for dbt (data build tool) processing. These models bridge the gap between hierarchical LDAP structures and relational data warehouse schemas by providing typed definitions for dimensions, facts, and dbt project configurations.
The system is organized into a primary facade class, FlextDbtLdapModels, which composes multiple specialized model namespaces including dimensions, schema definitions, project configurations, and execution results. This architecture ensures that data integrity is maintained throughout the extraction, transformation, and loading (ETL) pipeline.
Sources: src/flext_dbt_ldap/models.py:17-29, src/flext_dbt_ldap/protocols.py:11-19
Core Model Architecture
The model layer follows a composite pattern where the main DbtLdap namespace inherits from several functional model classes. This allows the service to handle everything from low-level LDAP query definitions to high-level dbt run statuses.
The diagram above illustrates the inheritance and composition of the dbt-ldap model namespace.
Sources: src/flext_dbt_ldap/models.py:17-29
LDAP Dimensions and Facts
Dimension models represent the canonical entities extracted from LDAP, such as users and groups. These models include built-in logic to map raw LDAP entry attributes (LDIF format) to typed Python objects.
User and Group Dimensions
The UserDimension and GroupDimension classes serve as the primary targets for LDAP data transformation. They include validation logic to ensure that mandatory business fields, such as user_id or group_id, are present before the data proceeds to the warehouse.
| Model | Field | Type | Description |
|---|---|---|---|
| UserDimension | user_id | str | Canonical user identifier (UID) |
email | str | None | Primary user email address | |
display_name | str | None | The name shown in UI components | |
manager_dn | str | None | DN of the user's manager | |
| GroupDimension | group_id | str | Canonical group identifier |
member_count | int | Count of direct group members | |
group_type | str | None | Classification of the group |
Sources: src/flext_dbt_ldap/_models/dimensions.py:16-118
Membership Facts
The MembershipFact model captures the relationship between users and groups, supporting both direct assignments and primary group markers.
class MembershipFact(FlextDbtLdapModelsShared.DbtSerializable):
user_dn: Annotated[str, u.Field(description="Member distinguished name")]
group_dn: Annotated[str, u.Field(description="Group distinguished name")]
membership_type: Annotated[str, u.Field(...)] = c.DbtLdap.DIRECT
is_primary: Annotated[bool, u.Field(...)] = False
Sources: src/flext_dbt_ldap/_models/dimensions.py:120-146
dbt Project and Schema Configuration
The models provide a structured way to define dbt schema.yml and dbt_project.yml components programmatically.
Schema Definitions
The DbtModelDefinition and DbtTestConfig classes manage the generation of dbt schema files. They track model versions and mapping configurations for column-level tests.
| Class | Attribute | Description |
|---|---|---|
DbtModelDefinition | models | Sequence of JSON mappings for dbt model definitions |
DbtTestConfig | columns | LDAP attributes keyed by name for dbt test application |
TransformationConfig | transformations | SQL expressions for column-level transformations |
Sources: src/flext_dbt_ldap/_models/schema.py:15-56
Project Configuration
The DbtProjectConfig model defines the directory structure and materialization strategies for a dbt project instance.
This flow shows how the project configuration influences the dbt environment setup.
Sources: src/flext_dbt_ldap/_models/configuration.py:15-53
Validation and Result Models
To ensure data quality, the system utilizes validation models that track the health of the LDAP data being processed.
Validation Metrics
The ValidationMetrics class quantifies the success of the data extraction and validation phase, calculating quality scores based on required attributes and DN validity.
- total_entries: Total LDAP entries evaluated.
- valid_entries: Entries satisfying all required attribute constraints.
- quality_score: A float representing the validation success rate.
- validation_passed: Boolean flag indicating if the score met the
min_quality_threshold.
Sources: src/flext_dbt_ldap/_models/results.py:16-36, src/flext_dbt_ldap/_models/schema.py:69-83
Execution Status
The DbtRunStatus and SyncResult models track the lifecycle of the ETL pipeline, providing visibility into which models were run and the number of entries processed.
Sources: src/flext_dbt_ldap/_models/results.py:38-66
Data Flow Protocol
The FlextDbtLdapProtocols defines how these models interact within the system's services. It establishes the contract for transforming raw LDAP entries into the UserDimension and GroupDimension models, and subsequently into dbt formats.
The diagram outlines the protocol-driven data flow from LDAP extraction to warehouse synchronization.
Sources: src/flext_dbt_ldap/protocols.py:53-76
The Data Schema Models are central to the flext-dbt-ldap project, providing the necessary type safety and structural definitions to transform complex, hierarchical LDAP data into reliable, warehouse-ready relational models using dbt.