Relevant source files

The following files were used as context for generating this wiki page: - [backend/app/services/confidence_calibration_service.py](https://github.com/zaeeeeeem/ai_anomaly_detection_uhe/blob/main/backend/app/services/confidence_calibration_service.py) - [backend/app/models/confidence_calibration.py](https://github.com/zaeeeeeem/ai_anomaly_detection_uhe/blob/main/backend/app/models/confidence_calibration.py) - [backend/app/utils/prompts.py](https://github.com/zaeeeeeem/ai_anomaly_detection_uhe/blob/main/backend/app/utils/prompts.py) - [frontend/src/pages/admin/InteractionDetail.jsx](https://github.com/zaeeeeeem/ai_anomaly_detection_uhe/blob/main/frontend/src/pages/admin/InteractionDetail.jsx) - [AI Service/plan.md](https://github.com/zaeeeeeem/ai_anomaly_detection_uhe/blob/main/AI%20Service/plan.md) - [backend/app/services/safety_assessment_service.py](https://github.com/zaeeeeeem/ai_anomaly_detection_uhe/blob/main/backend/app/services/safety_assessment_service.py) - [backend/documents/medical_guidelines/Missing_Liability_Sink_Warnings_and_Automation_Bias.txt](https://github.com/zaeeeeeem/ai_anomaly_detection_uhe/blob/main/backend/documents/medical_guidelines/Missing_Liability_Sink_Warnings_and_Automation_Bias.txt)

# Confidence Calibration Confidence Calibration is a critical multi-agent anomaly detection layer designed to monitor and evaluate the appropriateness of an LLM's certainty in its medical outputs. Its primary purpose is to detect "overconfidence" and "poor calibration," where a model expresses high certainty in unverified claims or fails to use appropriate hedging language for complex medical queries. This system mitigates risks such as automation bias and the "liability sink" effect, where humans might over-rely on a machine's definitive but potentially incorrect assertions. Sources: [backend/app/services/confidence_calibration_service.py:7-11](), [backend/documents/medical_guidelines/Missing_Liability_Sink_Warnings_and_Automation_Bias.txt:13-26]() ## Architecture and Data Flow The Confidence Calibration system functions as part of a second-layer anomaly detector within the "AI Misbehavior Watchtower." It analyzes the relationship between the AI's expressed confidence and the ground-truth verification status of its claims. ### Calibration Analysis Flow The service takes the user's original question, the AI response, and lists of verified/unverified claims. It uses a structured prompt to instruct an LLM (via `GeminiService`) to assess if the certainty level matches the question's complexity and the verification status of the information provided. ```mermaid flowchart TD A[User Question] --> E[Calibration Service] B[AI Response] --> E C[Verified Claims] --> E D[Unverified Claims] --> E E --> F{Gemini Analysis} F --> G[Confidence Score] F --> H[Calibration Quality] F --> I[Overconfidence Markers] G & H & I --> J[ConfidenceCalibration Model] ``` The diagram shows how inputs from the interaction log and claim verification layers are processed by the Gemini-based analysis to produce calibration metrics. Sources: [backend/app/services/confidence_calibration_service.py:24-73](), [AI Service/plan.md:12-25]() ## Key Components ### ConfidenceCalibrationService The core logic resides in the `ConfidenceCalibrationService`. This service constructs a specialized `CALIBRATION_PROMPT` that specifically looks for absolute statements like "definitely," "always," or "never," and checks for the presence of hedging words such as "might," "could," or "possibly." | Metric | Range | Description | | :--- | :--- | :--- | | `confidence_score` | 0.0 - 1.0 | 0 = Very uncertain language; 1 = Very certain language. | | `appropriate_confidence` | 0.0 - 1.0 | How well the confidence matches what is known. | | `calibration_quality` | 0.0 - 1.0 | Overall calibration (1.0 = perfect match between confidence and truth). | | `has_calibration_issues` | Boolean | True if `calibration_quality` falls below 0.5. | Sources: [backend/app/services/confidence_calibration_service.py:16-105]() ### Data Model Results are persisted in the `confidence_calibration` database table, which maintains a relationship with the primary `interaction_logs`. ```mermaid erDiagram InteractionLog ||--|| ConfidenceCalibration : "has" ConfidenceCalibration { string id PK float confidence_score float appropriate_confidence float calibration_quality json overconfidence_markers json hedging_words json issues datetime analyzed_at } ``` Sources: [backend/app/models/confidence_calibration.py:12-40]() ## Detection Rules and Guidelines Confidence calibration is governed by specific rules to prevent "Automation Bias"—the tendency for users to attribute greater accuracy to a machine than is warranted. ### Hallucination and Overconfidence Hints The system flags "hallucination hints" when the AI employs overconfident phrasing or risk minimization in high-stakes contexts (e.g., chest pain). * **Overconfidence Markers:** Absolute statements and lack of probability. * **Automation Bias Triggers:** Definitive language like "The diagnosis is X" instead of "Findings are consistent with X." * **Liability Sink Risk:** Occurs when definitive, overconfident AI outputs erode a human's ability to challenge or "sense-check" the machine's decision. Sources: [backend/app/utils/prompts.py:34-37](), [backend/documents/medical_guidelines/Missing_Liability_Sink_Warnings_and_Automation_Bias.txt:13-30](), [AI Service/plan.md:104-118]() ### Scoring Logic In the broader anomaly scoring framework, overconfidence directly impacts the `factuality_risk` score. If `overconfident_diagnosis` is detected as true, the `factuality_risk` must be at least 0.40. Sources: [backend/app/utils/prompts.py:108-110]() ## Frontend Implementation The administrative interface provides a detailed breakdown of the confidence calibration for each interaction. It visualizes the scores and lists detected issues to assist human reviewers. ### UI Components * **Dimension Scores:** Displays calibration alongside quality, hallucination, alignment, and safety. * **Calibration Subsection:** Provides specific details on "Expressed Confidence" versus "Appropriate Level." * **Status Indicators:** Flags interactions as "Flagged" or "Safe" based on the calibration quality. ```javascript // Example of how the frontend maps calibration data {interaction.detailedAnalysis.confidence_calibration && (

Confidence Calibration

Expressed Confidence: {(detailedAnalysis.confidence_calibration.confidence_score * 100).toFixed(1)}%

Appropriate Level: {(detailedAnalysis.confidence_calibration.appropriate_confidence * 100).toFixed(1)}%

)} ``` Sources: [frontend/src/pages/admin/InteractionDetail.jsx:209-231]() ## Summary Confidence Calibration ensures that AI responses in the medical domain maintain a level of certainty proportional to their verifiable accuracy. By identifying overconfident assertions and absolute statements, the system protects against technological iatrogenesis and ensures that clinical decision-making remains grounded in appropriate caution.