Optimized Summary System
Relevant source files
The following files were used as context for generating this wiki page:
The Optimized Summary System is a high-performance feature set designed to generate chat summaries for large volumes of Telegram messages. It improves upon the legacy summarization system by introducing parallel processing, context optimization, and hierarchical stages to handle high token counts and massive message datasets effectively.
This system manages the lifecycle of a summary request from initial message fetching through AI processing, selecting the most efficient strategy based on the data volume. It includes a robust fallback mechanism that reverts to the legacy system if optimized processing fails, ensuring continuous service availability.
Sources: README.md:10-18, src/features/summary/optimization/summary-controller.ts:40-60
System Architecture
The architecture relies on a centralized OptimizedSummaryController that orchestrates various specialized processors. It uses a StrategySelector to determine the data-handling path and a ContextOptimizer to manage token limits.
Component Overview
| Component | Function |
|---|---|
| OptimizedSummaryController | Manages sessions, fetches messages, and coordinates processing. |
| OptimizedStrategySelector | Selects between Direct, Parallel, or Hierarchical strategies based on message and token counts. |
| ContextOptimizer | Estimates token usage and chunks messages to fit model context windows. |
| ParallelProcessor | Uses concurrent Workers/Durable Objects to fetch and process message blocks. |
| HierarchicalProcessor | Performs two-stage summarization (preprocessing chunks then final summary) for very large datasets. |
| DirectProcessor | Processes messages in a single AI request for small to medium datasets. |
Sources: src/features/summary/optimization/index.ts:5-22, src/features/summary/optimization/summary-controller.ts:25-38
Request Flow
The following diagram illustrates how the system decides to process a summary request:
Strategy Selection Logic
The OptimizedStrategySelector determines the processing mode by evaluating the estimated message count and token volume. This ensures the system does not exceed Cloudflare Workers AI context limits while maximizing throughput.
Selection Criteria
- Direct: Selected when message counts and tokens are within a single model request limit.
- Parallel: Selected when message counts exceed the
minMessagesThreshold(default 100), allowing concurrent fetching. - Hierarchical: Selected when estimated tokens exceed the
chunkSizeThreshold(default 80,000 tokens).
Sources: tests/summary-optimization/basic-infrastructure.test.ts:88-105, README.md:120-130
Strategy Decision Diagram
Context Management
The ContextOptimizer ensures that message data fits within AI model constraints. It uses a tokenEstimationFactor (default 4) to approximate token counts from character lengths and implements "Smart Sampling" or "Sequential Chunking."
Key Responsibilities
- Token Estimation: Calculates the likely token count for an array of Telegram messages.
- Optimal Chunking: Splits large datasets into chunks that do not exceed
maxTokensPerRequest. - Smart Filtering: The controller filters out bot commands, mentions, and extremely short messages (less than 2 characters) before optimization begins to save context space.
Sources: src/features/summary/optimization/summary-controller.ts:466-490, tests/summary-optimization/context-optimizer.test.ts:70-100
Configuration
The system is configured via environment variables. If an invalid value is provided (e.g., MAX_WORKERS exceeding the hard limit), the system defaults to safe values or triggers the legacy fallback.
| Variable | Type | Default | Description |
|---|---|---|---|
SUMMARY_OPT_ENABLED | Boolean | true | Main feature flag for the optimized system. |
SUMMARY_OPT_MAX_WORKERS | Number | 5 | Maximum concurrent workers for parallel fetching. |
SUMMARY_OPT_MAX_TOKENS_PER_REQUEST | Number | 120000 | Max token budget for a single AI call. |
SUMMARY_OPT_HIERARCHICAL_ENABLED | Boolean | true | Enables two-stage processing for large data. |
SUMMARY_OPT_MIN_MESSAGES_THRESHOLD | Number | 100 | Minimum messages required to trigger parallel processing. |
Sources: README.md:113-130, tests/summary-optimization/basic-infrastructure.test.ts:45-65
Error Handling and Fallback
The OptimizedSummaryController implements a "Fail-Safe" pattern. Every optimized processing block is wrapped in a try-catch-finally structure. If an exception occurs, the system logs the error and immediately executes the legacy summarization function (summariseChatLegacy or summariseChatMessagesLegacy).
Fallback Procedure
- Catch error in
processOptimized. - Log error with session metrics and stack trace.
- Call
processLegacy. - If legacy succeeds, throw a
LEGACY_MESSAGE_SENTerror to prevent the controller from attempting to send a duplicate response.
Sources: src/features/summary/optimization/summary-controller.ts:114-125, src/features/summary/optimization/summary-controller.ts:427-444
Performance Monitoring
The system tracks detailed metrics for every session, including:
- fetchDuration: Time spent retrieving messages from KV/Durable Objects.
- processingDuration: Time spent in AI summarization.
- tokensUsed: Estimated total tokens processed.
- aiRequestsCount: Number of separate calls made to the AI provider.
Users can monitor these insights by filtering logs for "Optimized summary".
Sources: src/features/summary/optimization/summary-controller.ts:449-460, README.md:135-145
The Optimized Summary System provides a scalable foundation for chat analysis. By separating data retrieval from processing and dynamically choosing strategies, it maintains high performance even as chat history grows toward the 7-day TTL limit.