Context Compaction
Relevant source files
The following files were used as context for generating this wiki page:
Context Compaction is a system designed to manage long-running agent sessions by summarizing older parts of the conversation history. This prevents the agent from exceeding the LLM's context window, allowing for extended, complex tasks. The process involves identifying a "cut point" in the conversation, using an LLM to summarize all messages before that point, and replacing them with a single summary message. This preserves essential context while freeing up tokens.
The feature can be triggered automatically when the context usage reaches a configured threshold, or manually by the user. It is integrated into both the interactive Terminal UI (TUI) and the headless RPC mode.
Sources: packages/coding-agent/src/compaction.ts:3-8, packages/coding-agent/CHANGELOG.md:404-413
Core Logic
The core logic for context compaction resides in pure functions, which take session data and settings as input and produce a CompactionEntry without handling I/O directly. This logic is orchestrated by higher-level managers like the TuiRenderer or the RPC mode handler.
Sources: packages/coding-agent/src/compaction.ts:3-8
Configuration
Compaction behavior is controlled by the CompactionSettings interface.
| Setting | Type | Default Value | Description |
|---|---|---|---|
enabled | boolean | true | Toggles the entire compaction feature on or off. |
reserveTokens | number | 16384 | The number of tokens to keep reserved in the context window, triggering compaction when usage exceeds this buffer. |
keepRecentTokens | number | 20000 | The approximate number of tokens from recent messages to preserve verbatim (i.e., not summarize). |
Sources: packages/coding-agent/src/compaction.ts:20-28
Compaction Trigger
Compaction is triggered when the number of context tokens used approaches the model's context window limit, leaving a buffer defined by reserveTokens.
The shouldCompact function determines if compaction is necessary.
This diagram shows the decision flow for triggering compaction.
Sources: packages/coding-agent/src/compaction.ts:70-74
The total context tokens are calculated from the usage object of the last assistant message. The calculateContextTokens function sums the input, output, cacheRead, and cacheWrite tokens, or uses totalTokens if available.
Sources: packages/coding-agent/src/compaction.ts:34-37
Finding the Cut Point
Once compaction is triggered, the system must decide which messages to summarize and which to keep. The findCutPoint function identifies the index of the first message to keep.
The process is as follows:
- Identify all user messages ("turn boundaries") within the current active part of the session.
- Collect token usage data from all assistant messages, walking backward from the end of the session.
- Calculate the cumulative token difference from the newest message backward.
- When the token difference exceeds
keepRecentTokens, the previous assistant message is marked as the last one to keep. - The function then finds the user message (turn boundary) that occurs at or just before this assistant message. This index is the "cut point". All entries before this index will be summarized.
This ensures that a complete conversation turn (user message + assistant response) is always kept intact.
This diagram illustrates the logic for finding the cut point in the session history.
Sources: packages/coding-agent/src/compaction.ts:98-144
Summarization
The generateSummary function is responsible for creating the context summary by calling the LLM.
It uses a specific prompt to instruct the LLM to create a "handoff summary" for another LLM, focusing on key decisions, context, next steps, and critical data.
const SUMMARIZATION_PROMPT = `You are performing a CONTEXT CHECKPOINT COMPACTION. Create a handoff summary for another LLM that will resume the task.
Include:
- Current progress and key decisions made
- Important context, constraints, or user preferences
- Absolute file paths of any relevant files that were read or modified
- What remains to be done (clear next steps)
- Any critical data, examples, or references needed to continue
Be concise, structured, and focused on helping the next LLM seamlessly continue the work.`;
Sources: packages/coding-agent/src/compaction.ts:150-163
The summarization process is as follows:
- Messages to be summarized are transformed into a format the LLM can understand via
messageTransformer. - If a previous compaction occurred, its summary is prepended to the messages to provide historical context.
- The
SUMMARIZATION_PROMPTis appended. - The
completefunction from@mariozechner/pi-aiis called with the prepared messages to generate the summary.
Sources: packages/coding-agent/src/compaction.ts:168-193, packages/coding-agent/src/compaction.ts:221-240
Compaction Process Flow
The compact function orchestrates the entire process, from finding the cut point to generating the final CompactionEntry.
The following sequence diagram shows the high-level flow of the compact function.
Data Structures
CompactionEntry
When a compaction occurs, a CompactionEntry is created and saved to the session's .jsonl file. This entry contains the summary and metadata about the compaction event.
| Field | Type | Description |
|---|---|---|
type | string | Always "compaction". |
timestamp | string | ISO 8601 timestamp of when the compaction occurred. |
summary | string | The LLM-generated summary of the conversation history. |
firstKeptEntryIndex | number | The index in the session file of the first message that was not summarized. |
tokensBefore | number | The total context tokens before this compaction was performed. |
Sources: packages/coding-agent/src/compaction.ts:16-17, packages/coding-agent/src/compaction.ts:250-256, packages/coding-agent/test/compaction.test.ts:44-52
When a session is loaded, only the most recent CompactionEntry is used. The summary from this entry is used to create a synthetic user message that is placed at the beginning of the loaded conversation history.
Sources: packages/coding-agent/test/compaction.test.ts:182-194
TUI Integration
The context compaction feature is exposed in the interactive TUI through slash commands and automatic checks.
Sources: packages/coding-agent/src/tui/tui-renderer.ts
User Commands
Users can interact with the compaction system using the following commands:
| Command | Description |
|---|---|
/compact [instructions] | Manually triggers a compaction. Optional instructions can guide the summary. |
/autocompact | Toggles automatic compaction on or off. |
Ctrl+O | Toggles the expansion of the compaction summary view in the TUI. |
Sources: packages/coding-agent/CHANGELOG.md:404-413, packages/coding-agent/src/tui/tui-renderer.ts:1000-1006, packages/coding-agent/src/tui/tui-renderer.ts:1013-1018, packages/coding-agent/src/tui/tui-renderer.ts:490-503
Automatic Compaction
The TuiRenderer automatically checks if compaction is needed after each assistant message.
The following diagram illustrates the auto-compaction flow in the TUI.
RPC Mode Integration
Context compaction is also supported in RPC mode, enabling programmatic control and automation.
Manual Compaction
An RPC client can trigger compaction by sending a JSON command.
{
"type": "compact",
"customInstructions": "Focus on the file modifications."
}
The agent will perform the compaction and emit a compaction event back to the client upon completion.
Sources: packages/coding-agent/src/main.ts:1040-1067, packages/coding-agent/test/rpc.test.ts:139-141
Automatic Compaction
Similar to the TUI, RPC mode has an automatic compaction check that runs after each assistant message is saved. If shouldCompact returns true, the compaction is performed and a compaction event is emitted with an auto: true flag.
Sources: packages/coding-agent/src/main.ts:987-1025
Summary
Context Compaction is a critical feature for the pi-mono coding agent, ensuring that conversations can continue indefinitely without being limited by the LLM's context window. By intelligently summarizing past interactions while preserving recent ones, it maintains relevant context for long and complex tasks. The system is designed with a clean separation between pure logic and I/O-handling orchestrators, and it is seamlessly integrated into both the interactive TUI and the programmatic RPC interfaces.