Utility Scripts

Relevant source files

The following files were used as context for generating this wiki page:

• AI Service/plan.md
• backend/README.md
• backend/test_enhanced_endpoints.py
• backend/app/utils/prompts.py
• backend/documents/medical_guidelines/ESI_Decision_Point_Failure_Analysis.txt

Utility scripts in this repository provide essential administrative, testing, and operational support for the AI Anomaly Detection system. These tools facilitate system initialization, such as creating administrative credentials and ingesting medical guideline documents, and provide automated testing suites for verifying the integrity of enhanced API endpoints.

The scripts bridge the gap between the core backend logic and the operational requirements of the Medical Chatbot Watchtower. They ensure that the multi-agent orchestrator has access to the necessary data schemas and that the RAG (Retrieval-Augmented Generation) system is correctly seeded with medical safety guidelines.

Sources: backend/README.md, AI Service/plan.md:160-184

Administrative and Initialization Scripts

The project includes specific Python scripts designed for the initial setup of the environment. These scripts handle the creation of the primary administrative user and the population of the document store used for anomaly explanation.

Admin User Creation

The create_admin.py script is used to establish the first administrative account, which is required to access the Watchtower UI and the administrative API endpoints.

Parameter	Description
--email	The email address for the admin user.
--username	The chosen username for the admin account.
--password	The secure password for administrative access.

Sources: backend/README.md:10-12

Document Ingestion

The system relies on a RAG-backed explanation engine that requires a corpus of medical safety guidelines. Utility scripts (referenced as ingest_documents.py) are responsible for:

1. Crawling or reading local medical guidelines.
2. Chunking text into 400-800 token segments.
3. Generating embeddings and storing them in the medical_safety_guidelines Chroma collection.

Sources: AI Service/plan.md:160-180

Testing and Verification Scripts

To ensure the reliability of the anomaly detection layers, the repository provides automated testing scripts that simulate user interactions and verify API responses.

Enhanced Endpoint Testing

The test_enhanced_endpoints.py script provides a comprehensive suite for validating the "Multi-Dimensional Detection" capabilities of the backend. It tests endpoints related to metrics, anomaly breakdowns, and detailed interaction analysis.

The flow above represents the sequential execution of tests within the main() function of the enhanced testing utility.
Sources: backend/test_enhanced_endpoints.py:117-150

Smoke Testing

A shell script, phase6_smoke.sh, is used for high-level integration testing. This script requires the backend server to be running and verifies that the primary system components are communicating correctly.

Sources: backend/README.md:15-18

Data Simulation and Agent Prototyping

The project utilizes inline Python snippets for "Smoke Testing" specific agent logic, particularly for Phase 1 (Data & Logging) and Phase 2 (Rule-Based Scoring). These utilities allow developers to simulate high-risk prompts and verify if the WatchtowerOrchestrator correctly flags them.

Logic Verification Flow

This diagram illustrates how the test utility simulates an interaction to verify the LogAgent and Orchestrator integration.

Prompt Management Utilities

While not standalone scripts, backend/app/utils/prompts.py acts as a utility module that defines the strict JSON structures used by the Gemini-based agents. These templates are critical for ensuring that the scripts interacting with the AI service receive parseable data.

Template Name	Purpose	Key Output Fields
LEVEL_2_ANALYSIS_PROMPT	Feature extraction	topics, risk_context_flags, hallucination_hints
LEVEL_3_SCORING_PROMPT	Risk quantification	safety_risk, triage_risk, overall_anomaly_score
LEVEL_4_EXPLANATION_PROMPT	RAG explanation	risk_type, explanation, citations

Sources: backend/app/utils/prompts.py:7-140

Summary

The utility scripts in the ai_anomaly_detection_uhe repository provide the necessary scaffolding for administrative setup, document-based RAG seeding, and automated verification of the multi-agent detection pipeline. By leveraging these tools, developers can ensure the system is correctly initialized with medical guidelines and that the safety scoring logic remains consistent across updates.