Protocols & Interfaces
Relevant source files
The following files were used as context for generating this wiki page:
The Protocols & Interfaces module defines the structural contracts and service boundaries for the flext-dbt-ldap integration. It establishes how LDAP data is extracted, validated, and transformed into dbt-compatible formats within the FLEXT ecosystem. By utilizing Python's Protocol and @runtime_checkable decorators, the system ensures type safety and interoperability between LDAP directory services and data warehouse transformation layers.
These interfaces facilitate a modular architecture where specific implementations for data extraction, quality assurance, and dbt execution can be swapped or extended without breaking the core synchronization logic. The protocols extend base functionalities from flext-ldap and flext-meltano, inheriting standard service patterns while adding domain-specific requirements for dbt project management and LDAP schema mapping.
Sources: src/flext_dbt_ldap/protocols.py:1-25, src/flext_dbt_ldap/services/client.py:15-25
Service Protocol Architecture
The core architecture relies on FlextDbtLdapProtocols, which serves as a namespace for various domain-specific interfaces. These interfaces are categorized into functional areas such as Integration, Modeling, Transformation, and Quality.
Core Protocol Hierarchy
The following diagram illustrates the relationship between the dbt-ldap protocols and the base FLEXT ecosystem protocols.
Functional Protocols
The system defines specialized protocols to handle the lifecycle of LDAP data as it moves into dbt.
| Protocol Name | Purpose | Key Methods |
|---|---|---|
Dbt | Orchestrates dbt-specific operations. | run_dbt_models, compile_dbt_models, validate_dbt_project |
LdapIntegration | Manages the high-level sync pipeline. | extract_ldap_data, sync_ldap_to_warehouse |
Modeling | Defines dbt model structures from LDAP. | create_user_dimension, create_group_dimension |
Quality | Ensures data integrity before and after sync. | check_data_completeness, validate_ldap_schema_compliance |
Transformation | Handles attribute normalization. | normalize_ldap_attributes, enrich_ldap_data |
Sources: src/flext_dbt_ldap/protocols.py:27-130
Data Flow & Service Integration
The LdapIntegration protocol defines the flow of data from an LDAP source through a series of transformations and into a data warehouse. This process is orchestrated by service mixins like FlextDbtLdapClientMixin.
LDAP to dbt Pipeline Flow
The following sequence shows how the client service interacts with the defined protocols to execute a full synchronization pipeline.
Configuration & Result Models
Protocols depend on structured data models for inputs and outputs. These models define the schema for dbt project settings and the metrics returned by quality checks.
Dbt Project Configuration
The DbtProjectConfig model defines how the system interacts with the dbt project structure, including path definitions for models, macros, and snapshots.
- File Path:
src/flext_dbt_ldap/_models/configuration.py - Key Fields:
name,profile,model_paths,target_schema.
Sources: src/flext_dbt_ldap/_models/configuration.py:16-52
Quality & Performance Metrics
The ValidationMetrics and PerformanceAnalysis models provide standardized feedback for protocol methods.
| Model | Field | Type | Description |
|---|---|---|---|
ValidationMetrics | quality_score | float | Score based on valid vs total entries. |
ValidationMetrics | validation_passed | bool | True if score >= threshold. |
PerformanceAnalysis | execution_time | float | Time taken for sync/transformation. |
DbtRunStatus | models_run | Sequence[str] | List of dbt models executed. |
Sources: src/flext_dbt_ldap/_models/results.py:15-65
Synchronization & State Management
The FlextDbtLdapSyncMixin implements logic for incremental synchronization, relying on state persistence to track bookmarks (e.g., modifyTimestamp).
Incremental Sync Logic
The service uses a local state file to store bookmarks for different LDAP object types (users, groups).
Testing Protocols
Testing interfaces are provided in TestsFlextDbtLdapProtocols, which extend the standard protocols with database-specific interfaces like DbCursor and DbConnection. These allow for type-safe validation of the actual data written to the warehouse.
@runtime_checkable
class DbCursor(Protocol):
"""DB-API 2.0 cursor protocol for type-safe database operations."""
def execute(self, query: str | object, params: t.StrSequence | None = None) -> TestsFlextDbtLdapProtocols.DbCursor: ...
def fetchall(self) -> t.SequenceOf[tuple[FlextDbtLdapTypes.JsonValue, ...]]: ...
Sources: tests/protocols.py:28-40
Summary
The Protocols & Interfaces in flext-dbt-ldap provide a rigid yet extensible framework for managing the lifecycle of LDAP data within a dbt-managed data warehouse. By defining clear boundaries for extraction, transformation, quality assurance, and performance monitoring, the project ensures that LDAP-to-warehouse integrations remain maintainable and observable. Key components include the runtime-checkable protocols for dbt operations and the standardized result models that track sync status and data quality.