The "Fix Loop" Contributor Workflow
Relevant source files
The following files were used as context for generating this wiki page:The "Fix Loop" is an automated workflow designed to streamline the process of triaging and fixing failing scenarios within the Aldeci project. When a scenario fails, a developer can run the triage_and_fix.py script to automatically generate a dedicated git branch, a regression test, a patch file skeleton, and a markdown report with clear instructions. This process ensures a consistent and efficient bug-fixing methodology, particularly for issues related to the upstream Fixops project that Aldeci patches and extends.
The primary goal of this workflow is to reduce the manual overhead for contributors, enforce a standard procedure for creating fixes, and ensure that every fix is accompanied by a corresponding regression test. The generated artifacts guide the developer through creating a patch and submitting it for review, improving overall productivity and code quality.
Sources: tools/fixloop/triage_and_fix.py, README.md:1-5
Workflow Overview
The Fix Loop process is initiated by a developer with a result.json file from a failed test run. The triage_and_fix.py script then orchestrates the creation of all necessary files and git structures, leaving the contributor with a well-defined set of tasks to complete the fix.
The following diagram illustrates the end-to-end flow from triage to pull request.
Initiating the Fix Loop
To start the workflow, a developer must invoke the triage_and_fix.py script from the command line, providing the path to the result.json of a failed scenario and the path to the upstream file that needs to be patched.
Sources: tools/fixloop/triage_and_fix.py:73-76
Command-Line Interface
The script accepts the following arguments:
| Argument | Type | Description |
|---|---|---|
result | Path | The path to the result.json file from the failing scenario run. |
--upstream | str | The relative path to the upstream file in the Fixops project that requires a patch. |
Sources: tools/fixloop/triage_and_fix.py:74-76
An example invocation looks like this:
python tools/fixloop/triage_and_fix.py path/to/result.json --upstream path/to/upstream/file.py
The script parses these arguments to extract scenario details and determine the target for the patch.
# tools/fixloop/triage_and_fix.py:78-83
data = json.loads(args.result.read_text(encoding="utf-8"))
scenario_id = data["scenario"]
overlay = data["overlay"]
backend = data["backend"]
failure = json.dumps(data.get("invariant_report", {}))
Sources: tools/fixloop/triage_and_fix.py:78-83
Automated Scaffolding
Once initiated, the script performs several automated actions to prepare the development environment for the contributor.
Branch Creation
A new Git branch is created to isolate the fix. The branch is named using the pattern fix/<scenario_id>, where scenario_id is extracted from the input result.json. This is handled by the ensure_branch function.
Sources: tools/fixloop/triage_and_fix.py:23-24, tools/fixloop/triage_and_fix.py:85
Regression Test Generation
A placeholder regression test file is created at tests/scenarios/test_<scenario_id>_regression.py. This file contains a boilerplate test function that is intended to run the test matrix and confirm the fix. The generated test imports run_matrix_main from tools/e2e/run_matrix.py, linking the fix to the end-to-end testing framework.
# tools/fixloop/triage_and_fix.py:32-37
path.write_text(
"""from tools.e2e.run_matrix import main as run_matrix_main
def test_regression():
run_matrix_main()
""",
Sources: tools/fixloop/triage_and_fix.py:27-39
Patch File Skeleton
The script generates a patch file skeleton in the patches/fixops/ directory. The file is named <upstream_file_name>.<scenario_id>.patch and is placed in a subdirectory that mirrors the upstream project's structure. This file is pre-populated with a header that instructs the developer to add the source SHA of the upstream file and the patch content. The upstream file paths can be identified using the artifacts/upstream_symbol_index.json file.
# tools/fixloop/triage_and_fix.py:49-52
header = (
f"# Patch for {upstream_path}\n"
f"# Source-SHA: <fill-upstream-sha>\n"
f"# Scenario: {scenario_id}\n"
)
Sources: tools/fixloop/triage_and_fix.py:42-53, artifacts/upstream_symbol_index.json
Contributor Report Generation
Finally, a markdown report is generated at reports/fixloop/<scenario_id>.md. This report summarizes the failure and provides a checklist of steps for the contributor to follow to complete the fix.
The report is generated from the following template:
# Fixloop Report
- Scenario: {scenario_id}
- Overlay: {overlay}
- Backend: {backend}
- Failure: {failure_summary}
Steps:
1. Review generated test under `tests/scenarios/{scenario_id}_regression.py`.
2. Edit the patch skeleton in `{patch_path}` referencing the upstream file & SHA.
3. Update artifacts/upstream_map.json marking the patched file.
4. Commit changes and open a PR via the standard workflow.
Sources: tools/fixloop/triage_and_fix.py:12-21, tools/fixloop/triage_and_fix.py:60-71
The Contributor's Task
After the triage_and_fix.py script completes, the contributor's environment is fully prepared. The following sequence diagram outlines the manual steps required to complete the fix.
The instructions in the generated markdown report guide the developer through these final steps to ensure the patch is correctly implemented, documented, and submitted for review.
Sources: tools/fixloop/triage_and_fix.py:17-21
Conclusion
The "Fix Loop" workflow is a powerful developer productivity feature within the Aldeci project. By automating the tedious and error-prone aspects of setting up a bug fix environment, it allows contributors to focus on writing the patch itself. This structured process, which bridges failing tests directly to patch creation and regression testing, is essential for maintaining the stability and quality of Aldeci as it evolves alongside its upstream Fixops dependency.
Sources: tools/fixloop/triage_and_fix.py, README.md