Quality Gates

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• AGENTS.md
• pyproject.toml
• dbt_project.yml
• tests/unit/test_services.py
• tests/unit/_services_parts/data_quality.py
• tests/unit/test_dbt_models.py

Quality Gates in the flext-dbt-ldif project represent a multi-layered validation framework designed to ensure the integrity of LDIF data, the correctness of generated dbt models, and the compliance of code changes with universal engineering standards. These gates are implemented through automated testing, dbt data tests, and static analysis tools.

Sources: README.md:42-44, AGENTS.md:16-118, dbt_project.yml:51-54

Universal Engineering Gates

The project adheres to a "Zero-Tolerance / Strict-Total" policy for engineering conduct. These rules act as the primary quality gate for any AI agent or developer contributing to the codebase.

Mandatory Verification Flow

Every change must pass through a validation chain where "Done" is only achieved after objective evidence (command execution, exit codes, and output) is provided.

The diagram shows the mandatory sequence for code changes, emphasizing that any failure requires an immediate stop rather than a workaround.
Sources: AGENTS.md:21-25, AGENTS.md:54-58

Core Engineering Principles

The project enforces specific technical gates via its development environment:

• SSOT (Single Source of Truth): One authoritative source per fact; failures are triggered if SSOT is absent.
• Strict Typing: Use of the most restrictive types; Any and type suppressions (e.g., # type: ignore) are forbidden without documented justification.
• YAGNI: Removal of speculative parameters or dead branches.

Sources: AGENTS.md:86-101, pyproject.toml:138-164

Data Quality Assessment

Data quality gates are integrated into the FlextDbtLdifServiceMixin and exposed through the API to assess LDIF files before transformation.

LDIF Validation Logic

The system evaluates LDIF data based on several metrics, including total entry counts and a calculated quality score.

Component	Function/Method	Description
Validation Result	m.DbtLdif.LdifValidationResult	Data structure containing total_entries, quality_score, and validation_status.
Quality Service	run_data_quality_assessment	Triggers the parsing and validation workflow for a specific file.
Parsing Gate	parse_ldif_file	First gate; ensures the file is valid LDIF format before further assessment.

Sources: tests/unit/test_services.py:44-55, tests/unit/_services_parts/data_quality.py:27-50

The sequence diagram illustrates the delegation of data quality checks from the service layer to specialized validation clients.
Sources: tests/unit/test_services.py:44-70, tests/unit/_services_parts/data_quality.py:27-50

dbt Model Quality Gates

Quality gates for dbt transformations are defined in the dbt_project.yml and enforced through model-level configurations and data tests.

dbt Test Configuration

The project is configured to store failures and treat warnings as errors to ensure high-integrity transformations.

• Severity: Set to error for the dbt_ldif package.
• Failure Persistence: store_failures: true is enabled to allow auditing of records that fail quality checks.
• Materialization Strategy: Different materialization gates are applied per layer (e.g., staging uses view while foundational models use table).

Sources: dbt_project.yml:34-49, dbt_project.yml:51-54

Business Rule Validation

Dbt models must pass internal business rule checks before they are considered valid for generation.

# From tests/unit/test_dbt_models.py
def test_validate_business_rules_empty_sql(self) -> None:
    """Test business rules fail for empty sql_content."""
    model = m.DbtLdif.DbtModel(
        name="test_model",
        dbt_model_type="staging",
        ldif_source="ldif_entries",
        sql_content="  ", # Invalid: Empty SQL
        columns=[],
        dependencies=[],
    )
    result = model.validate_business_rules()
    assert result.failure

Sources: tests/unit/test_dbt_models.py:126-140

Static Analysis & Testing Gates

Automated gates are defined in pyproject.toml using make verbs: check, test, val, and docs.

Thresholds and Tools

Gate Type	Tool	Configuration/Threshold
Linting	ruff	target-version = "py313", select = ["ALL"]
Type Checking	mypy / pyright	strict_equality = true, reportUnnecessaryCast = "error"
Coverage	pytest-cov	fail_under = 20 (minimum precision)
Security	bandit / pip-audit	Part of the dev dependency group
Code Style	black	Standard formatting enforcement

Sources: README.md:42-44, pyproject.toml:57-124, pyproject.toml:132-136, pyproject.toml:225-245

Automated Test Suites

The project utilizes pytest with specific markers to categorize and gate different levels of implementation:

• unit: Isolated logic tests.
• integration: Cross-component workflow tests.
• e2e: Complete workflow from LDIF file to dbt output.

Sources: pyproject.toml:440-450, tests/unit/test_services.py:128-175

Conclusion

Quality Gates in flext-dbt-ldif ensure that every step of the LDIF transformation pipeline—from raw file ingestion to final data mart materialization—is verified against both technical engineering standards and specific data integrity rules. By integrating these gates into the dbt lifecycle and the developer's git workflow, the project maintains a high-quality baseline for enterprise LDAP data processing.

Sources: README.md:42-44, AGENTS.md:16-25, dbt_project.yml:51-54