Core Go Harness Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The Core Go Harness Architecture serves as a terminal-native Software Engineering (SWE) agent environment designed for extensibility and modularity. Its primary purpose is to provide a robust, stable core that enables AI agents to interact with development tools while maintaining cloud-agnostic portability across various backends like GCP, AWS, and Cloudflare.
The harness is built on the principle of a "Modular Monolith," prioritizing observability and operational simplicity. It integrates specialized components, such as the Rust-based swe_distiller, as extensions to handle heavy lifting like content extraction and document processing. This architecture ensures that the core remains lightweight while offloading complex logic to dedicated, high-performance modules.
Sources: AGENTS.md:275-285, extensions/swe_distiller/ARCHITECTURE.md:3-8
System Design Principles
The architecture follows a strict decision hierarchy to ensure maintainability and safety in production environments. Decisions are prioritized based on correctness, safety (blast radius protection), and observability.
Core Architectural Hierarchy
- Correctness & User Intent: The system must execute exactly what is requested.
- Safety & Blast Radius: Protection of secrets, data, and production systems is paramount.
- Observability: Failures must be visible, and diagnosis must be cost-effective.
- Operational Simplicity: The system must be operable by humans at any time.
Sources: AGENTS.md:22-38
Extension Integration: swe_distiller
A key component of the harness is the swe_distiller extension, a Rust-first extraction pipeline. It is orchestrated by the Go harness to convert URLs or documents into clean Markdown or JSON for agent consumption.
Pipeline Orchestration
The extraction pipeline follows a deterministic path that separates acquisition from extraction logic. This allows the harness to swap between heuristic-based extraction and LLM-assisted extraction seamlessly.
Module Layout and Responsibility
| Module | Responsibility | Source Reference |
|---|---|---|
main.rs | CLI Entry point and argument parsing | extensions/swe_distiller/src/main.rs:17-48 |
lib.rs | Pipeline orchestration and logic | extensions/swe_distiller/src/lib.rs:1-20 |
fetch.rs | URL acquisition, charset decoding, and browser fallbacks | extensions/swe_distiller/src/fetch.rs:31-70 |
extraction/llm.rs | LLM provider (Ollama/Jina) management | extensions/swe_distiller/src/extraction/llm.rs:20-50 |
markdown_ast.rs | DOM-walking and structural conversion to Markdown | extensions/swe_distiller/src/markdown_ast.rs:10-30 |
Data Flow and Acquisition
The harness handles acquisition via a multi-layered fetch system. It includes support for standard HTTP requests, retries with backoff, and a browser fallback for pages requiring JavaScript execution (e.g., Cloudflare challenges).
Fetch Sequence
Extension Configuration
The harness provides a CLI interface and environment variables to control the behavior of its extensions.
Distiller Options
| Parameter | Type | Description |
|---|---|---|
--mode | Enum | Output format: markdown or json. |
--llm | Boolean | Enables LLM-based extraction pipeline. |
--debug | Boolean | Enables structured JSON logging to stderr. |
SWE_DISTILLER_LLM_PROVIDERS | Env | Comma-separated list of LLM providers (e.g., jina,ollama). |
Sources: extensions/swe_distiller/src/main.rs:17-48, extensions/swe_distiller/README.md:25-40
Execution Safety and Verification
The architecture mandates a "Systems Preflight" before any destructive or production-impacting action. This includes deconstructing the request, performing tradeoff analysis, and determining the blast radius.
Safety Protocol
- Read-only inspection: Inspect state before modification.
- Reproduction: Ensure the state is reliably understood.
- Dry-run: Preview changes without applying them.
- Verification: Confirm the outcome matches intent.
- Rollback: Maintain a plan to revert changes.
Sources: AGENTS.md:46-60, AGENTS.md:195-208
Summary
The Core Go Harness Architecture is a modular system that combines a Go-based TUI and agent harness with high-performance Rust extensions. By isolating acquisition, extraction, and LLM orchestration, the system provides a flexible environment for SWE agents to operate safely and efficiently across diverse software environments.