Logging & Monitoring
Relevant source files
The following files were used as context for generating this wiki page:
- server/services/conversation/log.py
- server/services/execution/log_store.py
- server/services/conversation/summarization/working_memory_log.py
- server/services/conversation/summarization/summarizer.py
- server/agents/interaction_agent/runtime.py
- server/agents/execution_agent/runtime.py
- server/services/gmail/importance_watcher.py
The Logging & Monitoring system in OpenPoke is responsible for persisting multi-agent interactions, maintaining conversation continuity through summarization, and tracking background processes like email monitoring. It employs a structured XML-style tagging system for log entries, allowing for both human readability and machine parsing across different agent tiers.
This system ensures that the "Interaction Agent" has a consistent "working memory" and that "Execution Agents" maintain a clear audit trail of their tool calls and results.
Conversation Logging Architecture
The core of the interaction history is managed by the ConversationLog and WorkingMemoryLog. These components handle the storage of user messages, assistant replies, and internal orchestration markers.
Log Entry Structure
Logs are stored in a structured format using XML-like tags. Each entry typically includes a tag, a timestamp, and an encoded payload.
Sources: server/services/conversation/log.py:35-43, server/services/conversation/summarization/working_memory_log.py:27-32
| Tag | Purpose | Visibility |
|---|---|---|
<user_message> | Stores content sent by the human user. | Visible to User |
<poke_reply> | Stores responses generated by the Interaction Agent. | Visible to User |
<agent_message> | Reports results from Execution Agents back to the Interaction Agent. | System Only |
<wait> | Orchestration marker used to prevent redundant responses. | System Only |
<conversation_summary> | A distilled version of previous interactions. | System Only |
Working Memory & Summarization
To prevent token bloat in LLM contexts, the system utilizes a summarization service that compresses older log entries into a SummaryState. This state includes a text summary, the index of the last summarized entry, and a list of remaining "unsummarized" entries.
Sources: server/services/conversation/summarization/summarizer.py:84-118, server/services/conversation/summarization/working_memory_log.py:89-130
The diagram shows the flow of a log entry into the summarization pipeline.
Sources: server/services/conversation/log.py:127-148, server/services/conversation/summarization/summarizer.py:72-118
Execution Agent Journaling
Execution Agents maintain their own append-only journals to track specific tasks and tool invocations. This provides a detailed audit trail of what actions were taken behind the scenes.
Execution Log Store
The ExecutionAgentLogStore manages separate log files for each agent, identified by a slugified version of the agent's name. It records four primary types of events:
- Requests: Instructions received from the interaction agent.
- Actions: Specific tool calls initiated by the agent.
- Tool Responses: The data returned by the tools (e.g., Gmail API results).
- Agent Responses: The final summary sent back to the interaction agent.
Sources: server/services/execution/log_store.py:44-52, server/services/execution/log_store.py:108-124
Runtime Monitoring
During execution, the ExecutionAgentRuntime logs iterations of the tool-calling loop, including plan requests and successful/failed tool completions.
Sources: server/agents/execution_agent/runtime.py:48-115
This sequence illustrates how the Execution Runtime interacts with the Log Store during a task.
Sources: server/agents/execution_agent/runtime.py:53-120, server/services/execution/log_store.py:108-124
Background Process Monitoring
OpenPoke includes background watchers that proactively monitor external services and feed logs back into the system.
Important Email Watcher
The ImportantEmailWatcher polls Gmail at configured intervals (default 60 seconds) to detect urgent messages. It maintains a seen_store to avoid duplicate processing and uses an LLM classifier to determine importance.
Sources: server/services/gmail/importance_watcher.py:34-45, server/services/gmail/importance_watcher.py:155-175
- Polling Interval:
DEFAULT_POLL_INTERVAL_SECONDS = 60.0 - Lookback:
DEFAULT_LOOKBACK_MINUTES = 10 - Persistence: Stores seen message IDs in
data/gmail_seen.json.
Sources: server/services/gmail/importance_watcher.py:27-31
Implementation Details
File Paths
The system uses dedicated paths within the server/data/ directory for persistence:
- Conversation Logs:
server/data/conversation/poke_conversation.log - Working Memory:
server/data/conversation/poke_working_memory.log - Execution Logs:
server/data/execution_agents/{agent-name}.log - Seen Emails:
server/data/gmail_seen.json
Sources: server/services/conversation/log.py:25, server/services/conversation/summarization/working_memory_log.py:19, server/services/execution/log_store.py:16, server/services/gmail/importance_watcher.py:44
Concurrency Handling
All log stores utilize threading.Lock to ensure thread-safe writes to the log files, preventing data corruption during concurrent agent executions or simultaneous user requests.
Sources: server/services/conversation/log.py:48, server/services/execution/log_store.py:48, server/services/conversation/summarization/working_memory_log.py:38
Technical Summary
The Logging & Monitoring system provides a robust backbone for OpenPoke's multi-agent architecture. By combining structured file-based logging with LLM-powered summarization and proactive watchers, it enables complex, long-running interactions while maintaining high technical observability.