Feature: Patch Overlay System
Relevant source files
The following files were used as context for generating this wiki page:The Patch Overlay System is a strategic framework within the Aldeci project for managing modifications to its upstream dependency, the "Fixops" repository. Instead of forking the entire upstream project, this system allows for small, targeted, and validated patches to be applied. This approach is designed to minimize divergence from the upstream codebase, enforce code reuse, and maintain a clear, auditable trail of all modifications.
The system is built around a set of scripts, conventions, and automated workflows. It includes tools for scaffolding patch files during bug fixing, validating patch structure and content, and detecting when local code might be unnecessarily re-implementing functionality that already exists upstream. This ensures that any changes are deliberate, controlled, and easy to manage over time.
System Architecture and Core Principles
The architecture of the Patch Overlay System is centered on the relationship between the Aldeci repository and the upstream Fixops repository. Aldeci treats Fixops as a base and applies patches as an overlay.
Sources: tools/reuse/enforce_reuse.py:11, tools/fixloop/triage_and_fix.py:11
The following diagram illustrates the relationship between the Aldeci repository, the upstream code it depends on, and the patch overlay.
Upstream Code Baselining
To manage the relationship with the upstream code, Aldeci maintains an index of the Fixops repository's symbols (classes and functions). This index is stored in artifacts/upstream_symbol_index.json and serves as a manifest of the available upstream code.
Sources: artifacts/upstream_symbol_index.json, tools/reuse/enforce_reuse.py:84-86
Reuse Enforcement
A core principle of the system is to prevent the re-implementation of code that already exists in the upstream Fixops project. The tools/reuse/enforce_reuse.py script checks for this by comparing symbols defined locally within Aldeci against the upstream symbol index. If a local symbol has the same name as an upstream symbol but a different file path, it is flagged as a potential re-implementation. This guardrail encourages developers to use or patch the upstream code rather than creating duplicates.
Sources: tools/reuse/enforce_reuse.py:96-102
Patch Structure and Validation
All patches are stored in the patches/fixops/ directory and must adhere to a strict structure and set of validation rules. These rules are enforced by the validate_patch_structure function in tools/reuse/enforce_reuse.py.
Sources: tools/reuse/enforce_reuse.py:11, tools/reuse/enforce_reuse.py:61-72
Patch File Format
Each patch file must contain a specific header format to be considered valid. This header provides critical metadata about the patch's origin and purpose.
A patch skeleton is generated by the scaffold_patch function.
Sources: tools/fixloop/triage_and_fix.py:46-55
# Patch for {upstream_path}
# Source-SHA: <fill-upstream-sha>
# Scenario: {scenario_id}
--- a/path/to/original/file.py
+++ b/path/to/original/file.py
@@ -1,1 +1,1 @@
-original line
+new patched line
Sources: tools/fixloop/triage_and_fix.py:52-55, tools/reuse/enforce_reuse.py:20-21
Validation Rules
The following table details the validation rules applied to every patch file.
| Rule | Description | Source File |
|---|---|---|
| Header Presence | The first three lines of the patch must contain a header line starting with # Patch for . | tools/reuse/enforce_reuse.py:64-65 |
| Upstream SHA | The first five lines must contain a reference to the upstream commit SHA, starting with # Source-SHA:. | tools/reuse/enforce_reuse.py:66-67 |
| Line Limit | The number of changed lines (those starting with + or -) cannot exceed a maximum of 50. | tools/reuse/enforce_reuse.py:12,68-70 |
| Not Empty | The patch file cannot be empty. | tools/reuse/enforce_reuse.py:62-63 |
Automated Triage and Fix Workflow
The Patch Overlay System includes an automated workflow for triaging failing test scenarios and scaffolding the necessary components for a fix. This process is orchestrated by the triage_and_fix.py script.
Sources: tools/fixloop/triage_and_fix.py
The diagram below shows the automated workflow for creating a patch.
Workflow Steps
- Initiation: The workflow starts with a
result.jsonfile from a failing test run. A developer invokestriage_and_fix.py, passing the path to this JSON file and the relative path to the upstream file that needs patching.
Sources: tools/fixloop/triage_and_fix.py:80-82 - Branch Creation: A new Git branch is created with the format
fix/{scenario_id}.
Sources: tools/fixloop/triage_and_fix.py:27-28, tools/fixloop/triage_and_fix.py:88 - Test Scaffolding: A new regression test file is created at
tests/scenarios/test_{scenario_id}_regression.pyto ensure the fix can be validated.
Sources: tools/fixloop/triage_and_fix.py:31-44 - Patch Scaffolding: A patch skeleton is generated in the
patches/fixops/directory, mirroring the directory structure of the upstream file. The filename includes the original filename and the scenario ID (e.g.,{filename}.{scenario_id}.patch).
Sources: tools/fixloop/triage_and_fix.py:46-55 - Developer Action: The developer is responsible for writing the code changes in the patch file, finding and adding the correct upstream commit
Source-SHA, and updatingartifacts/upstream_map.jsonto register the new patch.
Sources: tools/fixloop/triage_and_fix.py:21-23 - Reporting: A markdown report is generated in
reports/fixloop/{scenario_id}.mdsummarizing the failure and the steps taken, including the path to the new patch file.
Sources: tools/fixloop/triage_and_fix.py:58-71
Key Components
The system relies on a few key scripts and artifacts to function.
| Component | Description | Source |
|---|---|---|
tools/fixloop/triage_and_fix.py | A script that automates the creation of a new fix branch, regression test, and patch skeleton based on a failing test scenario. | tools/fixloop/triage_and_fix.py |
tools/reuse/enforce_reuse.py | A CI script that enforces guardrails by validating patch structure and detecting re-implementation of upstream symbols. | tools/reuse/enforce_reuse.py |
patches/fixops/ | The root directory where all patch files for the upstream Fixops repository are stored. | tools/reuse/enforce_reuse.py:11 |
artifacts/upstream_map.json | A file that is updated as part of the fix-loop workflow, presumably to track which upstream files are patched. (Its exact structure is not defined in the provided files). | tools/fixloop/triage_and_fix.py:23 |
artifacts/upstream_symbol_index.json | A JSON artifact containing all the classes and functions from the upstream Fixops repository. It is used to detect symbol re-implementations. | artifacts/upstream_symbol_index.json, tools/reuse/enforce_reuse.py:84-86 |
reports/fixloop/ | A directory where markdown reports for each triaged scenario are stored. | tools/fixloop/triage_and_fix.py:12 |
Conclusion
The Patch Overlay System is a disciplined and highly automated approach for managing modifications to an upstream dependency. By favoring small, validated patches over a large-scale fork, it ensures that Aldeci can evolve while staying closely aligned with the upstream Fixops project. The system's guardrails and automated workflows promote maintainability, enforce code reuse, and provide a clear, auditable history of all changes, which is critical for long-term project health and security.