LLM Cascade & Model Fallbacks
Relevant source files
The following files were used as context for generating this wiki page:
The LLM Cascade and Model Fallback system is a core architectural component designed to provide high-latency content synthesis with high reliability. It orchestrates multiple Large Language Models (LLMs) to process YouTube transcripts into structured intelligence products using the Ultimate Content Intelligence & Implementation System (UCIS) framework. This system ensures that if a primary, cost-effective model fails or times out, the system automatically transitions to more robust backup models.
The system is a part of the Hybrid Edge Symphony architecture, where a Vercel-based "Bouncer" handles authentication and quotas, while a Cloudflare Worker manages the high-latency LLM streaming and the model cascade logic. This separation allows the project to exceed standard serverless execution limits while maintaining a cost-efficient and resilient analysis pipeline.
Sources: README.md, CLAUDE.md:14-20, GEMINI.md:124-135
LLM Cascade Architecture
The LLM Cascade follows a specific tiered sequence of models, starting with efficient or free models and falling back to more powerful, paid models like Anthropic's Claude Haiku 4.5. This sequence is managed by the LLMCascade service, which is categorized as a Domain Service within the project's Hexagonal Lite + DDD pattern.
Tiered Model Sequence
The cascade prioritizes models that minimize operational costs while fulfilling the requirements of the UCIS v5.1 framework.
| Tier | Priority | Model | Purpose |
|---|---|---|---|
| Tier 1 | Primary | nemotron-3-nano | Lead model for initial synthesis attempts. |
| Tier 2 | Fallback | Free Models | Interim fallbacks to maintain $0 cost queries. |
| Tier 3 | Ultimate | haiku-4.5 | High-reliability model used when other tiers fail. |
Sources: CLAUDE.md:38, GEMINI.md:131-140
Stratified Dual-Timeouts
To prevent hanging connections and ensure efficient fallback transitions, the system implements a stratified timeout strategy across the OpenRouter model sequence.
- Connection Handshake: A 3-second hard timeout to detect network faults or API unavailability early.
- Token Streaming Window: A 25-second maximum read limit on Vercel (matching serverless limits) or 90 seconds on Cloudflare Workers (for deep synthesis).
Sources: CLAUDE.md:8-11, GEMINI.md:92-95
Execution Flow and Error Handling
The system operates as a "blind, ruthless parser" under the UCIS v5.1 system. If a model in the cascade cannot fulfill a specific dimension due to lack of data in the transcript, it invokes the Insufficient Data Protocol rather than hallucinating or failing the entire request.
Content Synthesis Flow
The following diagram illustrates the flow from the initial request through the model cascade to the final persisted analysis.
The diagram shows the logic where the system first checks for a cached result before initiating the multi-tiered model cascade.
Sources: GEMINI.md:89-91, CLAUDE.md:38, conductor/tracks.md:27-30
Insufficient Data Protocol
When the transcript lacks the depth required for one of the 11 UCIS dimensions, the cascade logic adheres to strict constraints:
- No Inference: Forbidden from generating quotes or assuming context.
- Circuit Breaker: If a dimension cannot be fulfilled with high confidence, the model outputs exactly:
"[Insufficient data in source transcript to fulfill this dimension]". - Short-form Protocol: For videos under 180 seconds, models extract only visual/spoken elements and mark complex dimensions (like financial scenarios) as insufficient.
Sources: web/lib/prompts/ucis-v5.1.ts:46-72, supabase/migrations/20260611142500_add_prompt_config.sql
Security and Persistence
The output of the LLM cascade must be securely saved to the database. Because the analysis occurs in a Cloudflare Worker (outside the main Vercel app), a secure S2S (Server-to-Server) /persist flow is used.
HMAC Signature Verification
Data integrity is guaranteed through cryptographically signed writes. The Cloudflare Worker signs the analysis result using HMAC, which is then verified by the /api/analyses/persist endpoint in the web application.
This sequence ensures that only the authorized LLM streaming worker can persist analysis data to the primary database.
Sources: README.md:25-27, CLAUDE.md:14-20, conductor/tracks.md:31-32
Summary
The LLM Cascade and Model Fallback system ensures the hex-yt-intel platform provides high-density, 11-dimension content synthesis without being hampered by serverless timeouts or individual AI model failures. By utilizing a tiered approach (Nemotron → Free Fallbacks → Claude Haiku), stratified timeouts, and the Hybrid Edge Symphony architecture, the system achieves a balance between cost-efficiency and enterprise-grade reliability. The implementation of the Insufficient Data Protocol and HMAC-signed persistence further hardens the system against hallucinations and unauthorized data tampering.
Sources: GEMINI.md:124-140, web/lib/prompts/ucis-v5.1.ts:7-14, CLAUDE.md:36-40