Watchtower Core & Orchestrator
Relevant source files
The following files were used as context for generating this wiki page:
The Watchtower Core & Orchestrator serves as the central engine for an AI Misbehavior Watchtower, a second-layer multi-agent system designed to monitor LLM outputs in medical contexts. Its primary purpose is to ingest interaction logs, detect unsafe or misleading medical advice, generate RAG-grounded explanations for anomalies, and facilitate human-in-the-loop review.
Sources: AI Service/plan.md:1-10, frontend/src/components/admin/InteractionDetail.jsx:87-95
The system operates as a multi-stage pipeline that refactors standard LLM outputs into a structured safety analysis. It is designed to handle various risk categories including emergency triage failures, medication dosing errors, and medical hallucinations.
Sources: AI Service/plan.md:12-20, backend/app/utils/prompts.py:6-15
System Architecture & Pipeline
The Watchtower follows a 5-level processing pipeline that moves from raw data ingestion to expert human verification.
The diagram above illustrates the sequential data flow through the Watchtower processing stages.
Sources: frontend/src/components/admin/InteractionDetail.jsx:101-350, AI Service/plan.md:120-130
Level 1: Log Management
The LogAgent is responsible for ingesting, normalizing, and persisting interaction logs. It captures essential metadata such as model_name, user_id, and conversation_id.
| Field | Type | Description |
|---|---|---|
| interaction_id | UUID | Unique identifier for the log entry |
| prompt | string | The original user query |
| response | string | The AI-generated output |
| metadata | dict | Source info, language, and custom tags |
Sources: AI Service/plan.md:22-38, AI Service/watchtower/ui.py:66-80
Level 2 & 3: Analysis and Scoring
The MisbehaviorAnalysisAgent extracts features and risk flags, which the AnomalyScoringAgent then converts into quantitative risk scores (0.0 to 1.0).
- Analysis: Identifies emergency symptoms (chest pain, stroke signs), specialized populations (pediatrics, pregnancy), and dosing patterns.
- Scoring: Calculates specific risks for safety, triage, medication dosing, factuality, and anxiety.
Sources: AI Service/plan.md:133-165, backend/app/utils/prompts.py:53-65
Logic flow showing how logs are analyzed and scored before triggering the RAG system.
Sources: AI Service/plan.md:183-188, backend/app/utils/prompts.py:100-110
Orchestration Logic
The WatchtowerOrchestrator manages the execution of agents and handles the transition between different operation modes.
Operation Modes
The orchestrator supports two primary operational modes:
- Shadow Mode: Monitors and logs anomalies without blocking the output.
- Intercept Mode: If the
overall_anomaly_scoreexceeds a defined threshold (e.g., 0.75), the interaction can be flagged or blocked for immediate review.
Sources: AI Service/plan.md:290-300, AI Service/watchtower/ui.py:91-105
Core Data Structure
The ScoringRecord is the primary output used for decision-making within the orchestrator:
ScoringRecord = {
"interaction_id": "uuid",
"scores": {
"safety_risk": 0.0,
"factuality_risk": 0.0,
"triage_risk": 0.0,
"medication_dosing_risk": 0.0,
"overall_anomaly_score": 0.0
},
"flags": {
"missing_disclaimer": True,
"emergency_case": True,
"overconfident_diagnosis": True
}
}
Sources: AI Service/plan.md:58-75, backend/app/utils/prompts.py:126-155
Explainability and RAG System
For interactions flagged as anomalies, the orchestrator invokes the RAGSystem to generate human-readable explanations grounded in medical safety guidelines.
RAG Components
- Corpus: Seeded with authoritative sources like NHS Emergency Triage, WHO Medication Safety, and NICE/BNFC dosing guidelines.
- Vector Store: Uses Chroma with embeddings for medical safety guidelines, chunked into 400-800 tokens.
- Explanation Record: Contains the
risk_type(triage, dosing, disclaimer, etc.), a concise string explanation, and structured citations with relevance scores.
Sources: AI Service/plan.md:192-225, frontend/src/components/admin/InteractionDetail.jsx:290-315
Human Feedback Loop
The final stage of the orchestrator's workflow involves the FeedbackRecord, allowing expert reviewers to provide ground truth.
- Labels: SAFE, UNSAFE, BORDERLINE.
- Corrections: Reviewers can provide a
corrected_responsewhich is stored for future ML model refinement. - Analytics: The
WatchtowerUIprovides dashboards to visualize histograms of anomaly scores and frequency of specific risk flags over time.
Sources: AI Service/plan.md:88-97, AI Service/watchtower/ui.py:165-185, frontend/src/pages/admin/InteractionDetail.jsx:65-75
Summary
The Watchtower Core & Orchestrator provides a robust, multi-layered defense against LLM misbehavior in healthcare. By combining rule-based extraction, multi-dimensional scoring, RAG-grounded explanations, and human expert review, it ensures that AI-generated medical advice remains within safe and verified boundaries.
Sources: AI Service/plan.md:310-315, frontend/src/components/admin/InteractionDetail.jsx:90-95