Getting Started
Relevant source files
The following files were used as context for generating this wiki page:
The flext-dbt-ldif project provides dbt models and Python services for transforming LDIF (LDAP Data Interchange Format) data into structured SQL models. Its primary purpose is to automate the parsing, validation, and generation of staging and analytics layers within a dbt-managed environment, ensuring enterprise-grade data transformation for LDAP-sourced information.
Sources: README.md:1-12, pyproject.toml:28-35
Core Architecture and Components
The system is built as a Python-based orchestration layer that interacts with a dbt project. It uses a service-oriented architecture to manage the lifecycle of LDIF data processing.
System Flow Overview
The data flow starts with a raw LDIF file and moves through validation and model generation before final SQL transformation.
The workflow is orchestrated by the FlextDbtLdifServiceMixin.Service class, which manages the dependencies between parsing, model writing, and dbt execution.
Sources: src/flext_dbt_ldif/services/service.py:112-145, tests/unit/test_services.py:141-190
Primary Components
| Component | Description |
|---|---|
| Service Orchestrator | Manages the high-level workflow from file input to transformation results. |
| Unified Service | Responsible for generating dbt-specific SQL content for staging and analytics layers. |
| Client | Handles low-level operations like parsing LDIF files and interacting with the dbt CLI/API. |
| dbt Project | Contains the actual SQL templates, project configurations, and macros. |
Sources: src/flext_dbt_ldif/services/service.py:31-48, dbt_project.yml:1-50
Configuration and Setup
The project requires specific Python dependencies and dbt configurations to function correctly.
Environment Requirements
- Python: 3.13+
- Key Dependencies:
flext-core,flext-ldif,flext-meltano,pydantic. - dbt Versioning: Configured for
flext_ldifprofile.
Sources: pyproject.toml:34-40, README.md:5-7, dbt_project.yml:6-8
dbt Project Structure
The dbt_project.yml file defines the organizational structure for processed data:
| Layer | Materialization | Target Schema | Tags |
|---|---|---|---|
| Base | Table | ldif | ldif, enterprise |
| Staging | View | ldif_staging | staging, ldif |
| Intermediate | View | ldif_intermediate | intermediate, ldif |
| Marts | Table | ldif_marts | marts, ldif, business |
Sources: dbt_project.yml:25-50
Execution Workflow
Executing the workflow typically involves initializing the service and calling the complete workflow routine.
Workflow Sequence
The following sequence diagram illustrates the internal calls when processing an LDIF file.
Key Service Methods
parse_and_validate_ldif(ldif_file): Performs file parsing and checks data quality scores.generate_and_write_models(entries): Generates the SQL model files for dbt.run_complete_workflow(...): A high-level entry point that executes all stages in sequence.
Sources: src/flext_dbt_ldif/services/service.py:50-110
Project Standards
All development and execution within this repository must adhere to the Universal Agent Law defined in the project governance. This includes:
- Zero-Tolerance: Always fix root causes and remove superseded code.
- SSOT: Maintain a Single Source of Truth for configuration and contracts.
- Quality Gates: Use
makeverbs (check,test,val) to ensure standards before landing code.
Sources: AGENTS.md:10-50, README.md:31-35
Summary
flext-dbt-ldif streamlines the pipeline from raw LDAP exports to business-ready analytics tables. By integrating Python orchestration with dbt's transformation engine, it provides a robust framework for managing enterprise identity data. Users should start by configuring their dbt_project.yml variables (such as ldif_schema and database_name) before utilizing the Service class to process LDIF files.
Sources: dbt_project.yml:60-75, src/flext_dbt_ldif/services/service.py:31-48