Context Management
Relevant source files
The following files were used as context for generating this wiki page:
- plans/noesis-autonomous-compounding-implementation-plan.html
- plans/compounding-partnership-product-loop.html
- plans/noesis-high-leverage-correction-plan.html
- plans/noesis-first-complete-iteration.html
- AGENTS.md
- packages/domain/test/architecture.test.ts
- packages/tui/test/architecture.test.ts
- biome.json
Context Management in Noesis is the systematic process of assembling, freezing, and serving the semantic environment required for an LLM turn. The system utilizes a "Frozen Turn Plan" architecture to ensure that every turn executes against a byte-stable snapshot of prompts, tools, and retrieved history, preventing divergence between what is inspected and what is actually served to the model.
The core objective is to move from a mutable, inject-everything approach to an evidence-aware context compilation where every fragment has a documented provenance and strict token bounds. This management is handled primarily by the ContextCompiler and the WorkspaceStore, which coordinate the retrieval of history and the selection of active capability revisions.
Sources: plans/noesis-autonomous-compounding-implementation-plan.html:362-385, plans/noesis-high-leverage-correction-plan.html:208-223, AGENTS.md:31-35
Architecture and the Frozen Turn Plan
Noesis enforces a strict boundary between the mutable working files and the execution environment. Before a turn begins, the system freezes an ActivationSnapshot and a TurnIntelligencePlan. This plan pins specific file revisions, ensuring that even if a user edits a prompt during a turn, the executing model continues to use the byte-stable version recorded at admission.
Context Ownership Layers
The context is organized into six distinct ownership layers, ranging from hard-coded system rules to ephemeral turn data:
| Layer | Description | Authority |
|---|---|---|
| Protected Kernel | Code-owned authority, provenance, and safety rules. | Built-in / Code |
| User Constitution | Explicit, user-owned instructions that cannot be rewritten. | User Files |
| Learned Profile | Automatically evaluable style, reasoning, and workflow modules. | Noesis Learning |
| Capability Layer | Modules tied to specifically selected active capability revisions. | Capability Registry |
| Retrieved Context | Bounded history fragments and memory with exact citations. | SQLite / FTS |
| Turn Context | Current session data, active artifacts, and the user request. | Active Turn |
Sources: plans/noesis-autonomous-compounding-implementation-plan.html:439-447, plans/noesis-first-complete-iteration.html:764-775
Compilation Flow
The ContextCompiler is responsible for aggregating these layers into a single PromptSnapshot.
This diagram illustrates the transformation of a user request into a byte-stable FrozenTurnPlan containing a PromptSnapshot.
Sources: plans/noesis-autonomous-compounding-implementation-plan.html:388-405, plans/compounding-partnership-product-loop.html:380-388
Longitudinal Intelligence and Retrieval
Context is not merely a dump of history; it is a managed retrieval process. Noesis uses a hybrid search strategy to find relevant evidence while maintaining strict privacy and cost bounds.
- FTS5 & Embeddings: SQLite provides authoritative lexical search, while local embeddings provide semantic recurrence without exposing full history to external providers.
- Search Planner: An LLM-based search planner/reranker receives bounded candidates and returns a typed ranking with justification.
- Untrusted Context: Retrieved history is treated as untrusted context, distinct from protected instructions in the kernel or user constitution.
Citation and Provenance
Every piece of retrieved context must carry a session/message row reference or a file/evidence revision ID. This allows the system to "resolve" citations, turning opaque context back into its original source for inspection.
Sources: plans/noesis-autonomous-compounding-implementation-plan.html:492-503, AGENTS.md:28-30
Context Metrics (The Context Tax)
Noesis tracks the overhead of context management to prevent "accumulation" (buying performance with ever-growing prompts). This is measured via the Context Tax:
- Injected Context Tokens: Total tokens from history and memory.
- Prompt-Layer Bytes: Storage overhead of the static layers (Kernel, Profile, Capabilities).
- Marginal Input Tokens: The difference between the current served prompt and the baseline revision's prompt.
Sources: plans/noesis-high-leverage-correction-plan.html:145-155
Invariants and Safety Rails
Context management is governed by several architectural invariants enforced by tests and the AuthorityBoundary:
- Identity Verification: The application test ensures that the plan admitted by the
WorkspaceStoreis byte-for-byte identical to the plan handed to the execution adapter. - Context Redaction: Redaction creates a new input and is never reported as an exact replay.
- Isolation: Capability versions are never inferred from mutable working files; they must be pinned stable revisions.
- Noesis-Pi Boundary: The Pi substrate receives turn snapshots and typed tools but does not decide what they mean; Noesis manages the context and decision logic.
Sequence of context freezing ensuring that Pi substrate only sees data vetted and pinned by Noesis.
Sources: plans/noesis-high-leverage-correction-plan.html:246-260, AGENTS.md:23-35, packages/domain/test/architecture.test.ts:192-210
Implementation Details
The context management logic is distributed across specific packages to maintain boundaries:
| Package | Responsibility |
|---|---|
packages/context | Compiles layered prompts and manages token budgets. |
packages/workspace | Owns the persistence of revisions, artifacts, and citations. |
packages/domain | Defines the Zod schemas for TurnIntelligencePlan and PromptSnapshot. |
packages/runtime | Orchestrates the turn and triggers the compiler. |
Sources: plans/noesis-first-complete-iteration.html:734-750, packages/domain/test/architecture.test.ts:25-30