Testing & CI
Relevant source files
The following files were used as context for generating this wiki page:
M31 Autonomous (M31A) utilizes a multi-layered testing and continuous integration (CI) strategy to ensure the reliability of its terminal-native AI coding agent. The project emphasizes high test coverage, particularly for critical logic such as the task runner and rollback mechanisms, and employs automated validation scripts to gate releases and verify acceptance criteria.
The testing infrastructure includes unit tests, race detection, integration tests for full workflow cycles, and cross-platform build verification. All changes are governed by quality gates enforced through scripts like verify_v1.sh and validate-release.sh, ensuring that the "Execute-Verify" loop of the agent itself is mirrored in the project's own development lifecycle.
Sources: README.md:12-16, CONTRIBUTING.md:16-18, AGENTS.md:3-12
Testing Architecture
M31A's testing architecture is categorized into three primary levels: Unit Testing, Integration Testing, and Release Validation. The system is designed to be static and dependency-free (CGO_ENABLED=0), which simplifies the testing environment across different POSIX shells.
Testing Tiers and Targets
The project sets specific coverage targets for different components to maintain architectural integrity:
| Tier | Focus Area | Coverage Target |
|---|---|---|
| Overall | Project-wide source code | 75% |
| Critical Packages | pkg/taskrunner, pkg/bisect, pkg/rollback | 90% |
| Cross-Platform | Linux (amd64/arm64), Darwin (amd64/arm64), Windows (amd64) | 100% build success |
Sources: README.md:688-692, scripts/verify_v1.sh:34-45, CONTRIBUTING.md:18-20, AGENTS.md:17-21
Data Flow in Automated Verification
The following diagram illustrates how the verify_v1.sh script orchestrates different testing tools to validate the codebase:
Sources: scripts/verify_v1.sh:13-100, AGENTS.md:3-12
Integration and Workflow Testing
M31A includes complex integration tests that simulate a full seven-phase workflow. These tests ensure that the Engine correctly transitions between phases and interacts with the filesystem, Git, and LLM providers.
Workflow Simulation
The TestFullWorkflow in integration_test.go exercises the following flow:
- Initialize: Detects project type and initializes Git.
- Discuss: Simulates LLM clarifying questions and user answers.
- Plan: Generates and validates a task list in JSON format.
- Execute: Runs tasks using a mock tool dispatcher.
- Verify: Validates task outputs against acceptance criteria.
- Ship: Finalizes the session and archives the state.
Sources: internal/engine/workflow/integration_test.go:49-180
Multi-Turn Provider Mocking
To test the workflow without actual LLM costs, the project uses a multiTurnMockProvider that returns pre-defined responses mapped to specific workflow stages.
type multiTurnMockProvider struct {
callCount int
responses []string
}
// ChatCompletionStream returns responses[idx] based on callCount
Sources: internal/engine/workflow/integration_test.go:18-47
Continuous Integration and Release Validation
The project utilizes comprehensive validation harnesses to gate releases. These scripts go beyond standard testing to verify security, documentation consistency, and binary metadata.
Release Validation Harness (validate-release.sh)
This script performs 17 categories of checks, including:
- Binary Size: Ensures the final static binary remains under 60MB.
- Security Scanning: Checks for hardcoded API keys and verifies SSRF protections in
WebFetch. - Headless Mode: Validates that the
--promptflag works without the TUI. - Consistency: Compares phase counts and tool counts mentioned in documentation against the actual implementation.
Sources: scripts/validate-release.sh:58-75, scripts/validate-release.sh:230-265, scripts/validate-release.sh:314-345
Development Lifecycle Commands
Developers are expected to use a suite of make targets to maintain code quality:
| Command | Description |
|---|---|
make test | Runs race-enabled tests project-wide. |
make lint | Runs golangci-lint (vet, staticcheck, errcheck, ineffassign, unused). |
make check | Sequential run: fmt → tidy → vet → lint → test. |
make cover | Generates an HTML coverage report for visual analysis. |
make validate-release | Triggers the full release validation script. |
Sources: CONTRIBUTING.md:25-30, AGENTS.md:3-12, scripts/validate-release.sh:6-10
Security and Privacy Verification
M31A enforces a strict security model that is verified during CI:
- Static Analysis:
scripts/verify_v1.shchecks for "telemetry", "analytics", or "phone home" imports to ensure privacy. - Provider Isolation: Verification that no direct OpenAI or Anthropic SDKs are imported (only OpenRouter, Zen, and Nvidia gateways).
- Credential Safety: Checking that API keys are never written to disk in plaintext.
Sources: scripts/verify_v1.sh:86-90, scripts/validate-release.sh:314-328
M31A's Testing & CI infrastructure ensures that the agent's core promise—shipping verified code—is maintained by a project that itself adheres to the highest standards of automated verification and architectural discipline.