dbt Macros

Relevant source files

The following files were used as context for generating this wiki page:

• src/flext_dbt_ldap/_utilities/macros.py
• tests/test_macros.sql
• src/flext_dbt_ldap/protocols.py
• tests/generic/test_data_quality.sql
• src/flext_dbt_ldap/_models/configuration.py
• src/flext_dbt_ldap/_models/schema.py

The dbt macros in the flext-dbt-ldap project provide a set of reusable SQL and logic utilities designed to transform, normalize, and validate LDAP (Lightweight Directory Access Protocol) data within a data warehouse. These macros bridge the gap between complex LDAP hierarchical strings (Distinguished Names) and relational data models, enabling consistent data processing across staging, intermediate, and mart layers.

The project implements macros through two primary mechanisms: Python-based utility helpers that define the logic and SQL-based dbt macros that execute within the warehouse environment. These are used extensively for parsing Distinguished Names (DNs), converting generalized timestamps, and enforcing data quality standards.
Sources: src/flext_dbt_ldap/_utilities/macros.py, tests/test_macros.sql:1-5, src/flext_dbt_ldap/protocols.py:164-168

Core Transformation Macros

The transformation macros focus on extracting meaningful attributes from raw LDAP strings. Most LDAP entities are identified by a dn (Distinguished Name), which requires parsing to identify Relative Distinguished Names (RDNs) and hierarchy.

Distinguished Name (DN) Parsing

Macros such as parse_dn and parse_dn_component allow for the extraction of specific keys (like uid or cn) from a full DN string. These macros handle the comma-separated structure of LDAP paths.

The logic involves splitting the DN by commas and finding the specific component that matches the requested attribute (e.g., ou for Organizational Unit or cn for Common Name).
Sources: src/flext_dbt_ldap/_utilities/macros.py:68-81, tests/test_macros.sql:4-16

Normalization Utilities

LDAP attributes often arrive as arrays or single strings depending on the source extraction. Normalization macros ensure consistent types for downstream modeling.

Macro Name	Purpose	Input Type	Output Type
normalize_ldap_attribute	Extracts the first value from a list or returns the string.	str or list	str
normalize_array_field	Ensures field is a valid JSON array string.	str or null	JSON string
ldap_timestamp_to_timestamp	Converts GeneralizedTime (e.g., 20240101Z) to SQL Timestamp.	str	TIMESTAMP

Sources: src/flext_dbt_ldap/_utilities/macros.py:59-66, tests/test_macros.sql:32-48

Data Quality and Validation Macros

Validation macros are used in dbt tests to ensure the integrity of the LDAP data during the transformation pipeline. These are often called within generic tests or .yml schema definitions.

Validation Logic Flow

The validation macros typically return boolean values or filtered result sets to indicate whether an entry meets enterprise standards.

Key validation features include:

• validate_dn_format: Ensures a DN contains valid key-value pairs and isn't a malformed string.
• has_object_class: Verifies if an entry belongs to a specific LDAP objectClass (e.g., person or groupOfNames).
• is_user_active: Specifically checks the userAccountControl bitmask (checking if the 2nd bit is set) to determine account status.
  Sources: src/flext_dbt_ldap/_utilities/macros.py:53-57, tests/test_macros.sql:84-105, tests/generic/test_data_quality.sql:12-14

Programmatic Macro Generation

The system defines protocols for dynamically generating macros based on configuration. This allows the flext-dbt-ldap service to create dbt code on-the-fly to match specific LDAP schemas or business rules.

Macro Generation Protocols

The Macro protocol defines methods for creating various types of dbt macros:

• generate_ldap_source_macro: Creates sources.yml or source-calling macros based on DbtSourceDefinition.
• create_ldap_snapshot_macro: Automates the creation of dbt snapshots for Type 2 Slowly Changing Dimensions (SCD) on LDAP data.
• generate_ldap_transformation_macro: Generates macros based on TransformationConfig which maps source tables to specific column expressions.

Sources: src/flext_dbt_ldap/protocols.py:164-190, src/flext_dbt_ldap/_models/configuration.py:72-80, src/flext_dbt_ldap/_models/schema.py:35-46

Implementation Example: DN Attribute Extraction

The following Python logic serves as the backend for the extract_user_id_from_dn macro, which prioritizes multiple attributes to find a unique identifier.

@staticmethod
def extract_user_id_from_dn(dn: str) -> str | None:
    """Extract user ID from DN (tries uid, cn, samaccountname)."""
    for attr in c.DbtLdap.USER_ID_ATTRIBUTES:
        user_id = FlextDbtLdapUtilitiesMacros.parse_dn_component(dn, attr)
        if user_id:
            return user_id
    return None

Sources: src/flext_dbt_ldap/_utilities/macros.py:40-47

Summary

dbt Macros in this project are essential for handling the non-relational nature of LDAP data. They provide standardized methods for DN parsing, timestamp conversion, and membership counting, while also supporting dynamic generation of dbt assets via Python protocols. By encapsulating complex logic like bitmask checking (is_user_active) and recursive hierarchy validation, these macros ensure that the resulting data marts are accurate and business-ready.