Managing Gaps

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• artifacts/upstream_symbol_index.json
• artifacts/upstream_doc_index.json
• scripts/deep_review.py
• core/stage_runner.py
• docs/CODE_GAP_ASSESSMENT.md
• audit/GAPS.md
• reports/deep_review/gaps.json

The Aldeci project incorporates an automated gap analysis system to maintain high standards of code quality, documentation, and test coverage. This system programmatically identifies deficiencies in the codebase, such as missing documentation, absent test files, and lack of integration with continuous integration (CI) workflows.

The core of this functionality resides in a deep review script that analyzes the repository's structure and content. The findings are compiled into a machine-readable JSON report, providing developers with actionable insights to improve maintainability and robustness. This process is complemented by dedicated documentation, including a Code Gap Assessment and an Audit Gaps summary, which formalize the findings.

Sources: scripts/deep_review.py, artifacts/upstream_doc_index.json, README.md

Gap Detection Mechanism

Gap detection is performed by the detect_gaps function within the scripts/deep_review.py script. This function iterates through a collection of ModuleReport objects, each representing a Python module in the codebase, and applies a series of checks to identify specific types of gaps.

Sources: scripts/deep_review.py:118-139, artifacts/upstream_symbol_index.json

Detection Flow

The following diagram illustrates the logical flow of the detect_gaps function as it analyzes each module.

This diagram shows the process of checking for docstrings, corresponding test files, and CI mentions for each module.

Sources: scripts/deep_review.py:118-139

Types of Gaps

The script is configured to detect several specific categories of gaps, which are detailed in the table below.

Gap Type	Description	Detection Logic
Missing Docstring	The Python module does not have a module-level docstring.	The script checks if the module.docstring attribute is empty.
Missing Test File	A non-test source file does not have a corresponding test file in the tests/ directory.	It constructs an expected test file path (e.g., core/x.py -> tests/test_x.py) and checks for its existence.
Missing CI Mention	A test file exists but is not referenced in the primary CI configuration file (.github/workflows/qa.yml).	The script reads the content of the CI configuration and searches for the test file's path.

Sources: scripts/deep_review.py:122-137

The implementation of these checks can be seen in the detect_gaps function:

def detect_gaps(modules: list[ModuleReport], ci_config_path: Path) -> dict[str, list[str]]:
    """Detect missing docstrings, tests, and CI mentions."""
    gaps = {}
    ci_content = ci_config_path.read_text(encoding="utf-8") if ci_config_path.exists() else ""

    for module in modules:
        module_gaps = []
        if not module.docstring:
            module_gaps.append("Missing docstring")

        if module.path.startswith("tests/"):
            continue

        test_path = Path("tests") / f"test_{module.path.name}"
        if not test_path.exists():
            module_gaps.append(f"Missing test file for {module.path}")
        elif ci_content and str(test_path) not in ci_content:
            module_gaps.append(f"Missing CI mention for {test_path}")

        if module_gaps:
            gaps[module.path] = module_gaps
    return gaps

Sources: scripts/deep_review.py:118-139

Gap Analysis Output

The output of the gap analysis is a JSON file located at reports/deep_review/gaps.json. This file contains a dictionary where each key is the path to a module with identified gaps, and the value is a list of strings describing those gaps. This format is designed for both easy human review and further automated processing.

Sources: scripts/deep_review.py:151-152, artifacts/upstream_symbol_index.json

Sample Output

Below is a sample structure of the gaps.json file.

{
  "core/ai_agents.py": [
    "Missing CI mention for tests/test_ai_agents.py"
  ],
  "core/analytics.py": [
    "Missing CI mention for tests/test_analytics_store.py"
  ],
  "core/feedback.py": [
    "Missing docstring"
  ],
  "core/ssdlc.py": [
    "Missing test file for core/ssdlc.py"
  ]
}

Sources: reports/deep_review/gaps.json

Relation to SSDLC Stages

While the detect_gaps script is a standalone analysis tool, its purpose is closely aligned with the quality gates and evidence collection performed by the StageRunner class. The StageRunner orchestrates the Secure Software Development Lifecycle (SSDLC) by processing artefacts at different stages like build, test, and deploy.

For instance, the _process_test method in StageRunner evaluates code coverage and finding severity, which are quantitative measures of testing quality. Gap analysis provides a qualitative counterpart by ensuring that test files exist and are integrated in the first place. The _process_decision method synthesizes all stage outputs to render a final verdict, and a high number of identified gaps could implicitly influence this decision by highlighting areas of risk.

Sources: core/stage_runner.py:302-319, core/stage_runner.py:431-468