AI Explanations & Diagnostics

Relevant source files

The following files were used as context for generating this wiki page:

• services/ai_explain_service.py
• services/investigation_service.py
• services/code_execution_service.py
• webapp/templates/admin_observability.html
• repo_analyzer.py
• GUIDES/ARCHITECTURE.md

Introduction

The AI Explanations & Diagnostics system in CodeBot is a specialized framework designed to assist Site Reliability Engineering (SRE) teams in analyzing observability alerts and repository health. It leverages Large Language Models (primarily Anthropic's Claude) to transform raw metric spikes, error logs, and repository structures into actionable insights, root cause analyses, and suggested remediation steps.

The system acts as an intelligent middleware between raw monitoring data and the user interface, providing structured JSON-based diagnostic reports. It handles complex tasks such as PII masking, automated "Incident Story" generation, and multi-model fallback strategies to ensure high availability of diagnostic services even during provider outages.
Sources: [services/ai_explain_service.py:84-88](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L84-L88), [webapp/templates/admin_observability.html:200-220](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/webapp/templates/admin_observability.html#L200-L220), [GUIDES/ARCHITECTURE.md:120-135](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/GUIDES/ARCHITECTURE.md#L120-L135)

AI Explanation Service Architecture

The core logic resides in the AiExplainService, which orchestrates communication with AI providers to generate structured alert analyses. The service follows a strict pipeline of context sanitization, prompt construction, and result normalization.

Data Flow and Processing Pipeline

The following diagram illustrates how an observability alert is transformed into an AI-generated explanation.

The pipeline ensures that sensitive data like API keys, passwords, and session tokens are redacted before leaving the internal network.
Sources: [services/ai_explain_service.py:100-140](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L100-L140), [services/ai_explain_service.py:270-300](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L270-L300)

Core Components

Component	Description	Reference
AiExplainService	Primary coordinator for prompt building and provider communication.	[services/ai_explain_service.py:1](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L1)
Sanitizer	Redacts secrets (keys, fingerprints) and truncates logs using regex.	[services/ai_explain_service.py:100-120](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L100-L120)
Fallback Mechanism	Automatically rotates through models (e.g., Sonnet to Haiku) if the primary model fails.	[services/ai_explain_service.py:34-50](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L34-L50)
Incident Story	UI component for manual and automated documentation of diagnostics.	[webapp/templates/admin_observability.html:368-385](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/webapp/templates/admin_observability.html#L368-L385)

Observability Diagnostics

Diagnostics are integrated directly into the SRE dashboard, providing real-time analysis of anomalies, high error rates, and deployment spikes.

Incident Triage Flow

When a request ID is investigated, the system gathers metrics, logs, and traces to build a timeline for the root cause analysis.

Diagnostic Sections

The AI service generates three primary sections for every alert:

1. Root Cause: A factual, Hebrew-language summary of the underlying failure.
2. Actions: A list of up to 4 recommended remediation steps (e.g., "Analyze logs for endpoint X").
3. Signals: Key indicators extracted from the context, such as severity levels or deployment IDs.
   Sources: [services/ai_explain_service.py:84-88](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L84-L88), [services/ai_explain_service.py:325-345](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L325-L345)

Repository & Code Health Diagnostics

Beyond real-time observability, the system includes a RepoAnalyzer that performs static diagnostics on GitHub repositories to suggest architectural improvements.

Quality Metrics and Scoring

The diagnostics system calculates a "Quality Score" (0-100) based on several heuristics:

• Documentation: Presence and quality of README and LICENSE files.
• Automation: Detection of CI/CD workflows (GitHub Actions).
• Code Structure: Identification of "Large Files" (>500 lines) and "Long Functions" (>50 lines).
• Dependencies: Security scan for unpinned versions in requirements.txt or package.json.
  Sources: [repo_analyzer.py:440-480](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/repo_analyzer.py#L440-L480), [repo_analyzer.py:40-65](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/repo_analyzer.py#L40-L65)

Diagnostic Logic for Code Analysis

Security and Sanitization

A critical feature of the diagnostics system is the multi-layered sanitization process defined in services/ai_explain_service.py. This ensures compliance and security before data is sent to external AI providers.

Sanitization Rules

Rule Type	Logic	Action
Secret Masking	Regex for sk-, ya29., AIza	Replace with <secret>
Fingerprints	16+ character hex strings	Replace with <hash>
PII Keys	password, token, api_key	Replace value with <redacted>
Log Truncation	Max 4000 characters	Truncate and add ...
Metadata Limit	Max 40 items	Drop excess metadata
Sources: [services/ai_explain_service.py:100-160](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L100-L160)

# Implementation of PII masking for diagnostic context
def _mask_sensitive(text: str) -> str:
    def _kv_replacer(match: re.Match[str]) -> str:
        key = match.group(1)
        return f"{key}=<redacted>"

    sanitized = _SECRET_KEY_VALUE_RE.sub(_kv_replacer, text)
    sanitized = _SECRET_VALUE_RE.sub("<secret>", sanitized)
    sanitized = _HEX_FINGERPRINT_RE.sub("<hash>", sanitized)
    return sanitized

Sources: [services/ai_explain_service.py:102-114](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L102-L114)

Configuration

The AI diagnostics behavior is controlled through several environment variables.

Variable	Type	Default	Description
OBS_AI_EXPLAIN_MODEL	String	claude-sonnet-4-5-20250929	Primary model for diagnostics.
OBS_AI_EXPLAIN_MAX_TOKENS	Integer	800	Limits AI response length.
OBS_AI_EXPLAIN_TEMPERATURE	Float	0.2	Controls randomness (lower = more factual).
OBS_AI_EXPLAIN_TIMEOUT	Float	10.0	API request timeout in seconds.
Sources: [services/ai_explain_service.py:15-80](https://github.com/amirbiron/CodeBot/blob/1c758f824600c8d38c1354a25aa97d102a6caac2/services/ai_explain_service.py#L15-L80)

Conclusion

The AI Explanations & Diagnostics system serves as a robust intelligence layer within CodeBot, providing SREs with immediate context and actionable advice for system failures. By combining automated repository analysis with sophisticated AI-driven alert explanations—all while maintaining strict data privacy through sanitization—the system significantly reduces the "Mean Time To Resolution" (MTTR) for incidents.