Troubleshooting and Error Handling

Relevant source files

The following files were used as context for generating this wiki page:

• apps/api/errors.py
• apps/runtime/exceptions.py
• core/vector_store.py
• tools/fixloop/triage_and_fix.py
• tools/docgen/api_cli_map.py
• tests/test_api_dependencies.py
• tests/test_overlay_configuration.py

The Aldeci project employs a decentralized approach to error handling, integrating mechanisms at the API, application runtime, and tooling layers. While dedicated modules for exceptions exist (apps/api/errors.py, apps/runtime/exceptions.py), they are currently empty, indicating that error handling is implemented closer to where failures can occur. This includes using standard Python exceptions, FastAPI's built-in HTTPException for API responses, and custom tooling for managing test failures and documenting feature gaps.

This document provides an overview of the different error handling strategies, from API request validation to automated failure triage and reporting. Understanding these mechanisms is key to diagnosing issues, interpreting test failures, and contributing to the project.

API Error Handling

The Aldeci API, built with FastAPI, leverages the framework's standard dependency injection and exception handling to manage errors. Invalid requests, such as those with missing authentication, incorrect content types, or oversized payloads, are rejected before they reach the main business logic.

This is primarily demonstrated through the application's test suite, which asserts that specific HTTP error codes are returned under failure conditions.

Sources: tests/test_api_dependencies.py, tests/test_end_to_end.py

API Error Handling Flow

The following diagram illustrates how an incoming API request is validated, leading to either a successful response or an HTTP error.

Common API Errors

The tests confirm several validation checks that result in HTTP errors:

• Missing X-API-Key header: Results in an authentication error.
• Invalid JWT token: Results in an authentication error.
• Payload size exceeds limits: The validated_payload dependency will reject oversized requests.
• Incorrect Content-Type: Requests without the correct content type are rejected.

Sources: tests/test_api_dependencies.py:22-46

Custom Application Exceptions

While the central exception modules are empty, specific components within the application define their own custom exception classes to signal specific failure modes. This allows for more granular error handling within a particular domain.

VectorStoreError

A notable example is the VectorStoreError, used within the vector store components to indicate problems related to vector operations.

Sources: core/vector_store.py:28-29

# core/vector_store.py:28-29
class VectorStoreError(Exception):
    """Base exception for vector store operations."""

This base exception can be caught by callers to handle any issue originating from the vector store, regardless of the specific implementation (e.g., InMemoryVectorStore or ChromaVectorStore).

Automated Failure Triage

The project includes a dedicated script for handling and scaffolding fixes for failing test scenarios. The triage_and_fix.py tool automates the initial steps of the bug-fixing process, ensuring a consistent workflow for developers.

Sources: tools/fixloop/triage_and_fix.py

Triage Workflow

When a scenario test fails and produces a result.json file, a developer can run the triage script to automatically set up a dedicated environment for fixing the issue. The script performs the following actions:

1. Parses the failing result.json to extract scenario metadata.
2. Creates and checks out a new Git branch named fix/<scenario_id>.
3. Scaffolds a new regression test file under tests/scenarios/.
4. Scaffolds a patch file skeleton in the patches/fixops/ directory.
5. Generates a Markdown report in reports/fixloop/ detailing the failure and next steps.

The following flowchart shows the process executed by the triage_and_fix.py script.

Triage Report

The script generates a Markdown report to guide the developer. The report follows a consistent template.

Section	Description
Scenario	The unique identifier of the failing scenario.
Overlay	The configuration overlay used during the failed run.
Backend	The backend used during the failed run.
Failure	A JSON summary of the invariant report detailing the failure.
Next Steps	Instructions for the developer to review the test and edit the patch.

Sources: tools/fixloop/triage_and_fix.py:13-24

Reporting Feature Gaps

A unique aspect of troubleshooting in Aldeci is the systematic reporting of "GAPs"—features that are defined in the execution registry but are not yet implemented. The documentation generation tool api_cli_map.py is responsible for identifying and flagging these gaps.

Sources: tools/docgen/api_cli_map.py

Availability Badge

The _availability_badge function checks the available flag in the EXECUTION_REGISTRY for a given capability.

• If available is True, it returns a "✅ Available" badge.
• If available is False, it returns a "❌ GAP: {reason}" badge, making it immediately clear that the feature is not implemented.

This mechanism provides a clear, auto-generated status for every defined capability, preventing confusion and helping developers prioritize work.

# tools/docgen/api_cli_map.py:117-122
def _availability_badge(meta: Dict[str, Any]) -> str:
    if meta.get("available"):
        return "✅ Available"
    reason = meta.get("reason") or "Unavailable"
    return f"❌ GAP: {reason}"

Sources: tools/docgen/api_cli_map.py:117-122

Summary

Troubleshooting and error handling in Aldeci is a multi-layered process. At the highest level, the API uses standard HTTP status codes for request-related issues. Deeper in the application, custom exceptions provide context-specific error details. For development and testing, the triage_and_fix.py script provides an automated workflow to address failures. Finally, the project maintains transparency about its own completeness by systematically reporting unimplemented features as "GAPs" in its generated documentation. This comprehensive approach ensures that errors are handled gracefully at runtime and efficiently during development.