Unit Testing Strategy
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The unit testing strategy for the flext-dbt-ldap project is designed to validate the core logic of LDAP-to-warehouse synchronization, ensure the integrity of dbt macros, and maintain strict metadata standards. The strategy utilizes a combination of Python-based unit tests for service logic and SQL-based tests for dbt transformations and data quality.
The primary scope covers the validation of sync workflows, incremental state management (bookmarks), constant mappings, and dbt macro behavior. These tests are integrated into the project's quality gates as defined in pyproject.toml:574-604.
Core Components and Tooling
The testing infrastructure leverages pytest as the primary test runner, supported by several plugins to handle coverage, mocking, and environment management.
Key Tools and Dependencies
| Category | Tool / Package | Purpose |
|---|---|---|
| Test Runner | pytest | Main execution engine for unit and integration tests. |
| Mocking | pytest-mock | Used for stubbing service calls and filesystem operations. |
| Assertion | flext-tests (tm) | Provides fluent assertion syntax (tm.that(x, eq=y)). |
| Coverage | pytest-cov | Monitors code coverage with a minimum threshold of 20%. |
| SQL Testing | dbt | Executes generic and macro-specific SQL tests. |
Sources: pyproject.toml:68-124, pyproject.toml:210-213
Python Unit Testing Architecture
Python unit tests focus on service facades, specifically the FlextDbtLdapSyncMixin which manages data pipelines and state persistence.
Mocking and Pipeline Stubs
To isolate service logic from external LDAP systems and dbt runtimes, the strategy employs method monkeypatching. This allows developers to simulate successful or failed pipeline executions without requiring a live LDAP server.
State Persistence and Bookmark Validation
A critical part of the unit testing strategy is verifying that bookmarks (synchronization timestamps) are correctly generated, applied to filters, and persisted to the .flext_dbt_ldap_sync_state.json file.
- Incremental Logic: Tests verify that if a bookmark exists, the LDAP filter is appended with
(modifyTimestamp>=<bookmark>). - Type Safety: The system rejects non-string values in the sync state file, ensuring data integrity during initialization.
- Resilience: Tests verify that if state persistence fails (e.g., disk full), the service returns a failure result and does not leave a corrupt state.
Sources: tests/unit/test_dbt_services_sync.py:58-94, tests/unit/test_dbt_services_sync.py:118-132
SQL-Based Unit Testing
SQL unit tests validate the transformation logic within the dbt models and macros. These are categorized into macro tests and generic data quality tests.
Macro Verification
Macros are tested using static test data to ensure correct string manipulation and timestamp conversion.
| Macro | Test Focus |
|---|---|
parse_dn | Correct extraction of RDN, attribute, and parent DN. |
ldap_timestamp_to_timestamp | Handling of standard and fractional LDAP timestamps. |
normalize_array_field | Converting various inputs (single, array, null) to valid JSON arrays. |
validate_dn_format | Regex validation of Distinguished Names. |
Sources: tests/test_macros.sql:4-99
Data Quality Tests
Generic tests are applied to dbt models to enforce business rules and referential integrity across the warehouse schema.
Metadata and Constant Validation
Unit tests also ensure that the project's metadata (versioning) and internal constant mappings remain consistent.
- Version Alignment: Tests ensure
__version__strings match the__version_info__tuple exactly. - Constant Mapping: Tests verify that flattened constant surfaces (e.g.,
c.DbtLdap.CN) correctly point to their canonical definitions in the base LDAP module. - Model Mapping: Verification that LDAP entries are correctly mapped to
UserDimensionobjects with proper attribute assignment (e.g.,mailmapping toemail).
Sources: tests/unit/test_version.py:27-41, tests/unit/test_constants_flat_api.py:12-57
Summary
The flext-dbt-ldap Unit Testing Strategy ensures reliability through a multi-layered approach: Python unit tests for stateful sync logic, SQL tests for transformation macros, and strict metadata validation. By mocking the dbt pipeline and LDAP search operations, the test suite provides fast and deterministic feedback on the core logic governing LDAP data ingestion and transformation.