Testing Architecture & CI Integrations

Relevant source files

The following files were used as context for generating this wiki page:

• packages/domain/test/architecture.test.ts
• packages/tui/test/architecture.test.ts
• packages/runtime/test/turn-intelligence.test.ts
• packages/config/test/config.test.ts
• AGENTS.md
• plans/noesis-autonomous-compounding-implementation-plan.html
• plans/noesis-high-leverage-correction-plan.html

Introduction

The Noesis testing architecture is designed to enforce strict structural boundaries, verify the integrity of the autonomous learning loop, and ensure system reliability through a multi-layered verification strategy. It prioritizes "protection by construction," utilizing automated architecture tests to prevent leakage between internal modules, particularly between the generic runtime and third-party execution adapters like Pi.

The Continuous Integration (CI) pipeline integrates these tests to validate every change against a "causal spine." This ensures that adaptations made by the system are evidence-based, inspectable, and revertible. The architecture leverages Vitest for unit and integration testing, alongside specialized AgentHarness fixtures for credential-free validation of LLM-driven behaviors.

Architectural Verification

Noesis employs a robust set of automated architecture tests to maintain package isolation and enforce the "one authority per datum" rule. These tests scan imports and member access across the monorepo to prevent forbidden dependencies.

Boundary Enforcement

Tests ensure that high-level packages like runtime remain adapter-neutral, specifically forbidding imports from runtime-pi. Similarly, the TUI package is restricted to a narrow runtime port, preventing it from accessing durable internal state such as the ledger or workspace directly.

Key Verification Rules

Test Category	Target	Description
Pi Import Isolation	runtime-pi, tui	Pi-specific types/models are restricted to these two packages only.
Neutral Runtime	packages/runtime	Must remain independent of the Pi adapter to support multi-backend swaps.
TUI Isolation	packages/tui	Prevents the UI from owning durable policy or accessing low-level substrate.
Authority Mapping	packages/domain	Verifies that every persisted datum has exactly one declared authority.

Sources: packages/domain/test/architecture.test.ts:106-125, packages/domain/test/architecture.test.ts:261-268, AGENTS.md:25-27

Integrated Test Strategies

The project utilizes several testing modalities to verify complex behaviors ranging from configuration precedence to autonomous learning plans.

Turn Intelligence & Planning

Testing the "Turn Intelligence Planner" involves materializing immutable material (prompts, skills, routers) into a workspace and verifying that the planner admits the correct adaptation based on the context.

Configuration Precedence

Configuration tests verify a strict hierarchy of settings: CLI arguments override Environment variables, which override Config files, which override System defaults.
Sources: packages/config/test/config.test.ts:34-56

CI Pipeline & Verification

The CI environment is configured to run a complete suite of checks, denoted by the pnpm check command, which includes formatting, linting, type-checking, and testing.

Automated Gates

The CI rules enforce that the "Compounding Thesis" remains falsifiable through several specialized checks:

• Credential-Free Execution: Uses a Pi AgentHarness with a controlled provider to cover core loops without requiring paid API calls or network access.
• Identity Verification: The FrozenTurnPlan admitted by the WorkspaceStore must be byte-for-byte identical to the plan handed to the execution adapter.
• Fault Injection: Tests verify system resilience around admission failures, role timeouts, and activation publication.

Sources: plans/noesis-autonomous-compounding-implementation-plan.html:Task AC-14, plans/noesis-high-leverage-correction-plan.html:Verification, AGENTS.md:73-77

Falsifiable Compounding Metrics

CI monitoring tracks four key metrics to ensure the system's "Thesis" (that earlier work makes later work better) is measurable:

1. Served-revision win rate: Paired blind wins of served revisions over baselines.
2. Scope leakage rate: Frequency of adaptations served to unrelated contexts.
3. Context tax: Monitoring of marginal token costs for adaptations.
4. Correction recurrence: Rate at which corrections decrease after activation.

Sources: plans/noesis-high-leverage-correction-plan.html:The measurement that does not exist

Summary

The Testing Architecture in Noesis is not merely for bug detection but serves as a structural enforcement mechanism. By combining architecture tests with causal-spine verification in CI, the project ensures that its autonomous features remain under a "Protected Control Plane" where generated content can propose changes but cannot silently broaden its own authority. All information derived from AGENTS.md, implementation plans, and source test files.