High-Level Architecture

Relevant source files

The following files were used as context for generating this wiki page:

• AI Service/plan.md
• backend/app/utils/prompts.py
• backend/app/schemas/enhanced_detection.py
• frontend/src/components/admin/InteractionDetail.jsx
• backend/app/services/safety_assessment_service.py
• backend/test_enhanced_endpoints.py

The AI Anomaly Detection system, also known as the "AI Misbehavior Watchtower," is designed as a multi-layered, agentic framework to monitor LLM-generated medical advice. Its primary purpose is to ingest interaction logs, analyze them for safety risks and hallucinations, assign quantitative risk scores, and provide RAG-grounded explanations for flagged anomalies. The architecture facilitates a transition from research-paper analysis to a production-grade medical safety monitor.

The system is structured into five distinct phases: a Data & Logging Layer, a Rule-Based Analysis & Scoring Layer, a RAG-backed Explanation Engine, a Watchtower UI, and a Human Feedback Loop. This modular approach allows for real-time monitoring (Intercept Mode) or background auditing (Shadow Mode) of AI interactions to ensure compliance with medical safety guidelines.

Sources: AI Service/plan.md:1-12, frontend/src/components/admin/InteractionDetail.jsx:105-115

System Architecture Overview

The architecture follows a pipeline-based model where data flows through sequential stages of ingestion, feature extraction, scoring, and explanation.

Core Backend Components

1. Data & Logging Layer

The LogAgent serves as the entry point, normalizing and persisting interaction data. It supports ingestion from various sources (UI, API, or files) and stores them in a primary SQLite database with optional JSONL mirroring for auditing.

Component	Responsibility	Key Data Structures
LogAgent	Ingests, normalizes, and streams logs.	InteractionLog
Data Store	Persistent storage for logs and analysis results.	SQLite Tables, JSONL

Sources: AI Service/plan.md:15-110

2. Multi-Agent Analysis Pipeline

The system utilizes specialized agents to perform deep analysis of medical content.

• MisbehaviorAnalysisAgent: Extracts medical topics and identifies specific risk flags (e.g., missing disclaimers, medication dosing, pediatric cases).
• AnomalyScoringAgent: Assigns scores across five dimensions: quality, hallucination, alignment, safety, and confidence.
• SafetyAssessmentService: A specialized service focused on advice quality, emergency handling, and self-harm prevention.

Sources: AI Service/plan.md:115-185, backend/app/services/safety_assessment_service.py:18-30, backend/app/schemas/enhanced_detection.py:100-115

3. RAG-Backed Explanation Engine

When an anomaly score exceeds a defined threshold, the RAGSystem is triggered. It retrieves relevant context from a medical_safety_guidelines collection (stored in Chroma) to generate a concise, cited explanation of the safety violation.

Sources: AI Service/plan.md:195-260, backend/app/utils/prompts.py:126-140

Data Models and Schemas

The system relies on strict JSON schemas to ensure consistency between the AI agents and the database.

# Analysis Record Schema
AnalysisRecord = {
    "interaction_id": "uuid",
    "topics": ["chest_pain", "shortness_of_breath"],
    "risk_context_flags": {
        "missing_disclaimer": True,
        "gives_medication_dosing": False,
        "emergency_case": True,
        "triage_strength": "none|weak|adequate|strong"
    },
    "hallucination_hints": {
        "overconfident_phrasing": True,
        "risk_minimization": True
    }
}

Sources: AI Service/plan.md:33-58, backend/app/schemas/enhanced_detection.py:118-135

API and Integration Layer

The backend exposes enhanced administrative endpoints to allow the frontend and external tools to query metrics and detailed anomaly breakdowns.

Endpoint	Method	Description
/api/admin/metrics/enhanced	GET	Returns overall anomaly rates and dimension-specific averages.
/api/admin/analytics/anomaly-breakdown	GET	Provides a count of anomalies per category over a time period.
/api/admin/interactions/{id}/detailed	GET	Retrieves full multi-layer analysis for a specific interaction.

Sources: backend/test_enhanced_endpoints.py:53-120, backend/app/schemas/enhanced_detection.py:165-185

Frontend Presentation Layer

The "Interaction Detail" view in the admin dashboard visualizes the 5-level processing pipeline. Each level corresponds to a stage in the backend orchestrator:

1. Interaction Log: Displays the raw prompt and response.
2. Record Analysis: Shows extracted topics and boolean risk flags.
3. Scoring: Visualizes multi-dimensional risk scores (0-100%).
4. Explanation: Presents the RAG-generated critique and citations.
5. Human Review: Displays expert feedback, corrected responses, and comments.

Sources: frontend/src/components/admin/InteractionDetail.jsx:110-330

Conclusion

The architecture of the AI Anomaly Detection system provides a robust, multi-stage defense against unsafe LLM medical outputs. By combining rule-based heuristics, multi-agent scoring, and RAG-based explainability, it offers transparent and auditable oversight for AI-driven clinical advice.