Context Health & Quality Metrics
Relevant source files
The following files were used as context for generating this wiki page:
Context Health & Quality Metrics represent a specialized framework within the Token Optimizer project designed to quantify the efficiency, relevance, and "rot" of a Large Language Model (LLM) session's context window. This system moves beyond simple token counting by analyzing the semantic utility of context components, identifying waste in configuration files (like CLAUDE.md and MEMORY.md), and measuring the degradation of session quality over time. Sources: commands/quick.md, skills/token-coach/SKILL.md
The primary purpose of these metrics is to provide actionable insights that prevent performance degradation in long-running sessions. By identifying "stale" information, duplicated instructions, and bloated tool results, the system allows users to maintain a high-quality "Quality Zone" for peak model performance before context saturation occurs. Sources: skills/token-optimizer/references/agent-prompts.md, skills/token-coach/references/coaching-scripts.md
Metric Scoring & Thresholds
The system utilizes a composite Quality Score ranging from 0 to 100 to categorize the state of the current session context. This score determines the urgency of maintenance actions such as compaction or clearing. Sources: commands/quick.md:95-100, skills/token-coach/references/coaching-scripts.md:143-160
Quality Grade Categorization
| Score | Grade | Status | Recommended Action |
|---|---|---|---|
| 85 - 100 | A | Excellent | No action needed. Context is clean. |
| 70 - 84 | B | Good | Consider /compact to reclaim quality if bloat is detected. |
| 50 - 69 | C | Degraded | Performance at risk. Run /compact or full audit. |
| Below 50 | D/F | Critical | Heavy rot detected. Consider /clear or immediate audit. |
Sources: commands/quick.md:95-100, skills/token-coach/references/coaching-scripts.md:143-160
Architecture of Quality Measurement
The quality measurement process is driven by measure.py, a specialized script that interfaces with the runtime environment to collect snapshot data, historical trends, and real-time quality signals. Sources: skills/token-coach/SKILL.md:46-50, commands/quick.md:1-40
Data Flow for Quality Analysis
The following diagram illustrates how raw session data is processed into a quality report.
The flow demonstrates the authoritative detection of the runtime (Claude, Codex, or OpenCode) followed by the execution of measurement commands to generate JSON-formatted quality metrics. Sources: skills/token-coach/SKILL.md:46-75, commands/quick.md:41-55
Key Quality Indicators (KQIs)
Quality is determined by specific signals found within the context window. The system explicitly looks for "rot" or waste in several categories. Sources: skills/token-optimizer/references/agent-prompts.md, skills/token-coach/references/coaching-scripts.md:120-130
1. Context Rot & Information Loss
- Stale File Reads: Occurs when a file was read, subsequently edited, but never re-read by the model, leading to inaccurate context. Sources: skills/token-coach/references/coaching-scripts.md:121-125
- Bloated Tool Results: Large outputs from tool executions that are no longer referenced or needed in the active conversation. Sources: skills/token-coach/references/coaching-scripts.md:121-125
- Duplicate System Reminders: Accumulation of repetitive instructions or "permission denied" feedback that consumes tokens on every turn. Sources: skills/token-optimizer/references/implementation-playbook.md:100-115, skills/token-coach/references/coaching-scripts.md:121-125
2. Configuration Overhead
The system audits "always-loaded" files that consume a static percentage of every message.
- CLAUDE.md Waste: Verbose instructions (>50 lines) or tiered content that should be moved to on-demand files. Sources: skills/token-optimizer/references/agent-prompts.md:15-30
- Skill Menu Overhead: Each active skill adds ~100 tokens of frontmatter overhead. The system flags truncated descriptions (>1,536 characters) and unused skills. Sources: skills/token-optimizer/references/agent-prompts.md:100-130
3. Historical Trend Metrics
measure.py maintains a trends.db to track performance over time.
- Duration-Quality Correlation: Insights into how session length affects the quality score. Sources: skills/token-coach/SKILL.md:65-70
- Cache Hit Rate: Tracking the efficiency of prompt caching (Anthropic 5-min TTL prefixes). Sources: skills/token-coach/SKILL.md:65-70
- Compression Opportunity Gap: Measures the difference between current context size and what could be saved through optimized compaction. Sources: skills/token-coach/SKILL.md:65-70
Smart Compaction Workflow
To preserve quality during context-tight situations, the system utilizes "Smart Compaction." This workflow uses hooks to snapshot state before the native /compact mechanism summarily drops information. Sources: skills/token-coach/references/coaching-scripts.md:131-140, skills/token-optimizer/references/implementation-playbook.md:140-155
This diagram shows how Smart Compaction mitigates information loss by saving structured state to disk before lossy compaction and re-injecting it into the fresh context window. Sources: skills/token-coach/references/coaching-scripts.md:131-142, skills/token-optimizer/references/implementation-playbook.md:140-155
Implementation of Quality Audits
Quality audits are executed via parallelized agents in Phase 1 of the optimization flow. Each agent is responsible for a specific domain of context health. Sources: skills/token-optimizer/references/agent-prompts.md
| Auditor Agent | Model | Primary Quality Focus |
|---|---|---|
| CLAUDE.md Auditor | Sonnet | Size, tiered content, and cacheable prefix structure. |
| MEMORY.md Auditor | Sonnet | Duplication with global rules and operational history bloat. |
| Skills Auditor | Sonnet | Frontmatter overhead and truncated descriptions. |
| MCP Auditor | Sonnet | Tool Search status (deferred vs. eager loading) and description caps. |
| Settings Auditor | Sonnet | Hook configuration and file exclusion (permissions.deny) efficiency. |
Sources: skills/token-optimizer/references/agent-prompts.md:1-250
Conclusion
Context Health & Quality Metrics provide a data-driven approach to maintaining the efficiency of LLM interactions. By quantifying "rot" and providing a standardized Quality Score, the system enables users to transition from reactive context clearing to proactive context management. Significant savings (up to 25%+) are achieved not just through token reduction, but by extending the period of high-quality model reasoning through Smart Compaction and behavioral optimization. Sources: plugins/token-optimizer/skills/token-optimizer/SKILL.md:12-15, skills/token-coach/references/coaching-scripts.md:160-170