Prompt Caching & Context Compression
Relevant source files
The following files were used as context for generating this wiki page:
- agent/prompt_builder.py
- website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/developer-guide/prompt-assembly.md
- website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/user-guide/features/context-files.md
- agent/subdirectory_hints.py
- agent/learn_prompt.py
- agent/context_compressor.py
- agent/conversation_compression.py
Prompt Caching and Context Compression are core mechanisms in the Hermes Agent designed to maximize token efficiency and maintain high performance during long-running sessions. The system organizes prompt data into stable, cached layers and dynamic, ephemeral layers to ensure that the AI model can reuse previously processed information without redundant computation.
These mechanisms address the physical limits of Large Language Model (LLM) context windows by compressing large data sources, truncating oversized files, and progressively discovering relevant project context only when it is needed.
Prompt Assembly Architecture
Hermes Agent assembles the system prompt in a specific order to optimize for prompt caching. By placing stable information at the beginning of the prompt and dynamic information at the end, the agent minimizes cache misses.
Layered Structure
The prompt is divided into two primary categories: the cached system prompt layer and the ephemeral layer.
| Layer | Source / Content | Purpose |
|---|---|---|
| Agent Identity | SOUL.md or DEFAULT_AGENT_IDENTITY | Defines the persona and core behavior. |
| Tool Guidance | Internal instructions | Directs how the agent uses available tools. |
| Memory | MEMORY.md | Injects cross-session facts and user preferences. |
| User Profile | USER.md | Provides stable user-specific information. |
| Skills Index | skills/ directory | Lists available specialized capabilities. |
| Project Context | .hermes.md, AGENTS.md, etc. | Provides repository-specific instructions. |
| Ephemeral Data | Session overrides, prefill messages | Context only relevant to the current turn. |
Sources: website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/developer-guide/prompt-assembly.md:14-36, agent/prompt_builder.py
Prompt Data Flow
The following diagram illustrates how different context sources are processed and ordered into a single system prompt.
The "Cached Prefix" remains stable across turns to ensure the provider can reuse the processed tokens.
Sources: website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/developer-guide/prompt-assembly.md:100-115
Context Compression & Truncation
To stay within token limits and maintain performance, the agent applies strict compression and truncation rules to all project context files.
Truncation Strategy: 70/20 Head-Tail
When a context file (such as AGENTS.md) exceeds the character limit, the agent performs a specialized truncation. It preserves the beginning of the file (where high-level summaries usually reside) and the end of the file (where recent or specific instructions are found).
- Max Characters: 20,000 (approximately 7,000 tokens).
- Head Retention: 70% of the allowed limit.
- Tail Retention: 20% of the allowed limit.
- Middle: Replaced with a marker indicating truncation and advising the agent to use file tools for full access.
Content Processing Logic
Every context file undergoes a security scan and a character count check before it is injected into the prompt.
Progressive Context Discovery
Rather than loading every file in a large repository at startup, which would bloat the prompt and break caching, Hermes Agent uses "Progressive Discovery."
Subdirectory Hints
The SubdirectoryHintTracker monitors tool calls (like read_file or terminal) to identify when the agent enters a new directory. When the agent accesses a path, the tracker:
- Extracts the directory path from the tool arguments.
- Traverses up to 5 parent directories to find relevant context files (
AGENTS.md,CLAUDE.md, etc.). - Injects the discovered context into the tool result for that specific turn.
This ensures that specific instructions for a frontend/ or backend/ folder are only visible when the agent is actually working in those areas.
Sources: website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/user-guide/features/context-files.md:23-45, agent/subdirectory_hints.py
Project Context Priority
Hermes loads only one primary project context file per session to avoid conflicting instructions. It follows a strict priority hierarchy:
.hermes.md/HERMES.md: Highest priority; discovery walks up to the git root.AGENTS.md: Secondary; standard agent instruction file.CLAUDE.md: Compatibility layer for Claude Code users..cursorrules: Compatibility layer for Cursor IDE users.
Sources: agent/prompt_builder.py, website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/developer-guide/prompt-assembly.md:65-80
Security and Integrity
All files included in the prompt undergo a security scan for "Prompt Injection." The scanner blocks content containing patterns such as:
- Instruction overrides (e.g., "ignore previous instructions").
- Hidden HTML comments or invisible Unicode characters.
- Credential stealing attempts (e.g.,
curlcommands targeting API keys).
Sources: website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/user-guide/features/context-files.md:80-95, agent/learn_prompt.py
Summary
Prompt Caching and Context Compression enable Hermes Agent to operate effectively in complex codebases. By utilizing a stable layered prompt assembly, 70/20 head-tail truncation, and progressive context discovery, the agent balances the need for deep project knowledge with the technical constraints of LLM context windows and token costs.