Evidence and Retrospective System

Relevant source files

The following files were used as context for generating this wiki page:

• src/evidence/manager.ts
• src/config/evidence-schema.ts
• src/services/evidence-summary-service.ts
• src/services/evidence-service.ts
• src/hooks/system-enhancer.ts
• src/tools/write-final-council-evidence.ts

The Evidence and Retrospective System is a core architectural component of the OpenCode Swarm designed to provide deterministic, verifiable proof of work for every task and phase. It acts as a quality gate, ensuring that no task is considered complete without accompanying artifacts such as code reviews, test results, and structural diffs. This system enables the swarm to maintain high standards of code quality and architectural coherence through continuous feedback loops.

The system also handles retrospectives, which are specialized evidence artifacts captured at the end of each phase. These retrospectives store execution metrics, lessons learned, and error taxonomies, which are then injected back into the system via hooks to inform and improve the behavior of agents in subsequent phases or projects.

Sources: src/services/evidence-summary-service.ts, src/hooks/system-enhancer.ts:163-180

Architecture and Data Flow

The evidence system operates by aggregating individual evidence entries into task-specific bundles stored within the .swarm/evidence/ directory. Each bundle is a JSON artifact containing a collection of evidence objects that adhere to strict schema definitions.

Evidence Processing Flow

The following diagram illustrates how evidence is generated by agents, persisted by the manager, and eventually summarized for project oversight.

Evidence is categorized into various types, with review and test being non-negotiable requirements for task completion. The system enforces size limits on these artifacts, capping JSON bundles at 500KB and task-related binary/patch data at 20MB.

Sources: src/config/evidence-schema.ts:4-6, src/services/evidence-summary-service.ts:60-65, src/evidence/manager.ts

Data Models

The system relies on a discriminated union of evidence types, ensuring that each entry contains the necessary metadata while allowing for type-specific details like test pass rates or security findings.

Base Evidence Structure

All evidence entries share a common foundation:

Field	Type	Description
task_id	string	The identifier of the task (e.g., "1.1").
type	enum	The category of evidence (review, test, etc.).
timestamp	string	ISO 8601 datetime of creation.
agent	string	The name of the agent that produced the evidence.
verdict	enum	pass, fail, approved, rejected, or info.
summary	string	Human-readable summary of the artifact.

Sources: src/config/evidence-schema.ts:31-40

Specialized Evidence Types

The system supports 13 distinct evidence types, including:

• ReviewEvidence: Includes risk levels (low to critical) and a detailed list of issues with file/line references.
• TestEvidence: Tracks tests_passed, tests_failed, and specific failure messages.
• RetrospectiveEvidence: Captures phase-level metrics like total_tool_calls, coder_revisions, and error_taxonomy.
• QualityBudgetEvidence: Monitors complexity deltas, duplication ratios, and test-to-code ratios against defined thresholds.

Sources: src/config/evidence-schema.ts:43-160

Retrospective Injection System

A critical feature of the system is the "Tiered Retrospective Injection." This mechanism uses system hooks to feed lessons from past work into the currently active agent's prompt.

Injection Tiers

1. Tier 1 (Intra-project): Direct lookup for the previous phase in the same plan. Used for Phase 2+. It injects outcomes, top rejection reasons, and lessons learned to prevent repeating mistakes.
2. Tier 2 (Cross-project): Historical lookup for Phase 1 of a new project. It scans the workspace for retrospectives from recent prior projects (within 30 days) and carries forward high-level user directives.

Sources: src/hooks/system-enhancer.ts:163-270

Summary and Reporting Service

The EvidenceSummaryService provides deterministic aggregation of evidence. It produces both machine-readable JSON artifacts and human-readable markdown summaries to help human operators or high-level agents (like the Architect) understand project status.

Completion Logic

A task is considered isComplete: true only if:

1. Its status is marked as completed in the plan.
2. It contains all REQUIRED_EVIDENCE_TYPES (currently review and test).

Sources: src/services/evidence-summary-service.ts:60-65, src/services/evidence-summary-service.ts:130-150

Error Taxonomy

Retrospectives classify failures into a specific taxonomy to improve automated debugging:

• planning_error
• interface_mismatch
• logic_error
• scope_creep
• gate_evasion

Sources: src/config/evidence-schema.ts:124-132

Final Council Gate

At the end of a project, the system uses the write_final_council_evidence tool to synthesize a project-scoped final verdict. This requires a quorum of five specific agents: critic, reviewer, sme, test_engineer, and explorer. This tool persists a final-council.json artifact which serves as the ultimate certification of the project's completion and quality.

Sources: src/tools/write-final-council-evidence.ts:15-30, src/tools/write-final-council-evidence.ts:98-115

The Evidence and Retrospective System ensures that every action taken by the swarm is recorded, verified, and used as a learning opportunity, creating a self-improving autonomous development environment.