Memory Management
Relevant source files
The following files were used as context for generating this wiki page:
Memory Management in pi-digby refers to the multi-layered system used to persist context, preferences, and knowledge across agent conversations. The system utilizes file-based persistence, structured session context, and semantic indexing to ensure the agent maintains continuity in both global and channel-specific environments.
The architecture distinguishes between "Working Memory" (Markdown files), "Conversation Context" (structured message logs), and "Settings" (JSON configuration). This ensures that while the agent's immediate context window is limited, it can retrieve and update long-term information stored on persistent storage such as AWS EFS.
Sources: src/persistence/memory.ts:1-10, src/agent/prompt.ts:133-145, CLAUDE.md:1-15
Persistent Memory Layers
The system organizes memory into a hierarchical structure that allows for shared global knowledge and isolated channel interactions.
Global Workspace Memory
Global memory is stored in MEMORY.md at the root of the workspace directory. It contains information applicable across all channels, such as project info, general skills, and shared user preferences.
Sources: src/persistence/memory.ts:18-28, src/agent/prompt.ts:133-138
Channel-Specific Memory
Each channel has its own directory containing a local MEMORY.md file. This file tracks ongoing work, channel-specific decisions, and contextual information relevant only to that specific conversation thread.
Sources: src/persistence/memory.ts:31-41, src/agent/prompt.ts:133-138
Working Memory Loading
The loadMemory function aggregates these layers into a single text block for the agent's system prompt.
The diagram shows the sequential check for global and channel-specific Markdown files to build the agent's working memory context.
Sources: src/persistence/memory.ts:11-45
Conversation Context and History
Beyond Markdown-based memory, the system manages structured history to maintain conversation flow.
Context vs. Logs
The system maintains two distinct files per channel to handle history:
| File | Format | Purpose |
|---|---|---|
context.jsonl | Structured JSONL | API-ready messages including tool results for the current LLM context window. |
log.jsonl | Human-readable JSONL | Long-term history of user messages and final bot responses (excludes tool results). |
SYSTEM.md | Markdown Log | Logs environment modifications like package installs and config changes. |
| Sources: src/persistence/context.ts:1-12, src/agent/prompt.ts:145-152 |
Context Management Logic
The AgentSession manages the context.jsonl file, which is synced with the human-readable log.jsonl during the runner setup. This ensures that messages received while the agent was offline are integrated into the active session.
Sources: src/agent/setup.ts:136-160, src/persistence/context.ts:15-18
This sequence illustrates how history is restored from disk into the live agent state before a run begins.
Sources: src/agent/setup.ts:162-177
Settings and Environment Persistence
Settings are managed via a SettingsManager which persists data to a settings.json file in the workspace directory.
Workspace Settings Storage
The WorkspaceSettingsStorage class implements a locking mechanism to handle reads and writes to the global settings file. This storage tracks bot-wide configurations that persist across container restarts.
Sources: src/persistence/context.ts:25-50
Environment and MCP Persistence
System-level persistence is handled via AWS EFS mounts. Key directories like .pi (containing MCP configs and auth tokens) and .gitconfig are symbolically linked from the persistent EFS volume to the container's root directory during entrypoint execution.
Sources: entrypoint.sh:5-20
Semantic Memory (QMD)
The project incorporates qmd (likely a vector-based or semantic indexing tool) to handle large-scale memory collections.
Collection Registration
On startup, several directories are registered as semantic collections:
/data/memory: Global workspace memory and notes./data: Filtered for channel-specificmemory/**/*.mdfiles./data: Filtered for skill definitions (SKILL.md).
Sources: entrypoint.sh:31-45
Indexing Process
The system automatically updates and embeds these collections in the background to facilitate semantic retrieval.
Sources: entrypoint.sh:48-49
Memory Usage in Prompts
The memory retrieved from files is directly injected into the system prompt, providing the LLM with its "Working Memory."
// src/agent/prompt.ts:133-143
## Memory
Write to MEMORY.md files to persist context across conversations.
- Global (${workspacePath}/MEMORY.md): skills, preferences, project info
- Channel (${channelPath}/MEMORY.md): channel-specific decisions, ongoing work
Update when you learn something important or when asked to remember something.
### Current Memory
${memory}
Sources: src/agent/prompt.ts:133-143
Conclusion
Memory Management in pi-digby is a hybrid system combining human-readable Markdown for explicit persistence, structured JSONL for conversation continuity, and semantic indexing for long-term retrieval. By layering global and channel-specific storage on AWS EFS, the system ensures the agent remains context-aware across distributed environments and restarts.