Entry Parsing & Utilities
Relevant source files
The following files were used as context for generating this wiki page:
The Entry Parsing & Utilities module provides a robust framework for extracting, validating, and transforming LDAP entries into structured formats suitable for dbt processing. It serves as the bridge between raw LDAP directory data and the analytical data warehouse, ensuring data quality through schema validation and typed dimensions. Sources: src/flext_dbt_ldap/services/client.py, src/flext_dbt_ldap/_utilities/integration.py
Core Transformation Engine
The transformation logic is encapsulated within the FlextDbtLdapUtilitiesIntegration class, which handles the conversion of Ldif.Entry objects into domain-specific dimensions and facts. This process involves identifying entry types (users, groups, or organizational units) and mapping their attributes to dbt-compatible models. Sources: src/flext_dbt_ldap/_utilities/integration.py:22-30, src/flext_dbt_ldap/services/client.py:165-179
Dimension Transformation Flow
The system utilizes a generic transformation flow to process batches of LDAP entries into typed dimensions.
The diagram shows the logic within the shared _transform_entries_to_dimensions method used for users and groups. Sources: src/flext_dbt_ldap/_utilities/integration.py:115-141
Key Utilities Methods
| Method | Purpose | Source File |
|---|---|---|
transform_users | Converts LDAP entries into UserDimension objects. | integration.py:73-81 |
transform_groups | Converts LDAP entries into GroupDimension objects. | integration.py:30-38 |
transform_memberships | Extracts relationship data to create MembershipFact objects. | integration.py:40-66 |
_map_entry_attributes | Maps raw LDAP attributes to dbt-defined field names. | client.py:146-163 |
LDAP to dbt Mapping Logic
The mapping process translates hierarchical LDAP attributes into flat structures. This includes handling Distinguished Names (DNs), object classes, and multi-valued attributes. Sources: src/flext_dbt_ldap/services/client.py:146-163, tests/test_macros.sql:1-15
Attribute Mapping Strategy
- DN Resolution: The
dnattribute is extracted as a string for every entry. - Configured Mapping: Attributes defined in
settings.ldap_attribute_mappingare renamed to their dbt counterparts. - Unmapped Attributes: Any remaining attributes are preserved with their original names.
- Value Selection: For multi-valued LDAP attributes, the first value is typically selected for the dbt model.
Sources: src/flext_dbt_ldap/services/client.py:149-162
SQL-Level Utilities (Macros)
For transformations occurring within the data warehouse, dbt macros are provided to parse and validate LDAP strings.
The diagram illustrates how SQL macros transform raw string data in the warehouse. Sources: tests/test_macros.sql:1-60
Validation and Data Quality
Before transformation, entries undergo quality checks to ensure integrity. The FlextDbtLdapClientMixin performs validation against a min_quality_threshold. Sources: src/flext_dbt_ldap/services/client.py:115-144
Quality Metrics
The system tracks several metrics during the validation phase:
- Total Entries: Count of all entries in the extraction.
- Valid DNs: Entries containing a non-empty Distinguished Name.
- Valid Entries: Entries containing all attributes defined in
required_attributes. - Quality Score: Ratio of valid entries to total entries.
Sources: src/flext_dbt_ldap/services/client.py:126-133
Incremental Sync Utilities
The FlextDbtLdapSyncMixin provides utilities for incremental data synchronization. This relies on "bookmarks" to track the last successful synchronization point, typically using the modifyTimestamp attribute. Sources: src/flext_dbt_ldap/services/sync.py:18-40, src/flext_dbt_ldap/services/sync.py:133-145
Sync State Management
The state is persisted to a local JSON file (.flext_dbt_ldap_sync_state.json) containing timestamps for different sync keys (e.g., "users", "groups"). Sources: src/flext_dbt_ldap/services/sync.py:168-175
Conclusion
The Entry Parsing & Utilities system ensures that LDAP data is not only extracted but also validated and normalized before it enters the dbt modeling pipeline. By combining Python-based transformation logic with SQL-level macros and incremental state management, it provides a complete toolkit for managing LDAP identities within a modern data stack.