Adding New LLM Providers
Relevant source files
The following files were used as context for generating this wiki page:
The hex-yt-intel platform utilizes a sophisticated LLM Cascade architecture to ensure high-performance, cost-effective, and reliable YouTube content intelligence. This system relies on a multi-cloud hybrid flow where the core reasoning is distributed across various providers via OpenRouter to handle high-latency synthesis without hitting serverless execution limits.
Integrating a new LLM provider involves updating the Hybrid Edge Symphony (ADR 005), which coordinates requests between Vercel and Cloudflare Workers. The system is designed for high-performance streaming, using a stratified dual-timeout architecture to manage connection handshakes and token streaming windows across different model tiers.
Sources: GEMINI.md:126-136, README.md:20-27, CLAUDE.md:14-23
LLM Cascade Architecture
The project employs a hierarchical model cascade where specific models are prioritized based on performance and cost. Currently, the cascade lead is nemotron-3-nano, with fallback sequences leading eventually to Haiku 4.5.
Model Selection & Hierarchy
Providers are integrated into a triple-redundant hybrid model. When adding a new provider, it must be positioned within the existing fallback sequence defined in the Architectural Decision Records (ADRs).
The diagram shows how requests flow through the Vercel bouncer and then to the Cloudflare-based LLM Cascade for processing.
Sources: CLAUDE.md:14-23, CLAUDE.md:44-50, GEMINI.md:126-136
Strategic Dual-Timeouts
Every integrated provider must adhere to the stratified timeout architecture to prevent hanging connections in serverless environments.
| Timeout Type | Duration | Purpose |
|---|---|---|
| Connection Handshake | 3 Seconds | Detects network faults or provider downtime early. |
| Token Streaming (Vercel) | 25 Seconds | Hard limit for Vercel execution window. |
| Token Streaming (Worker) | 90 Seconds | Extended horizon for complex synthesis in Cloudflare. |
Sources: CLAUDE.md:17-21, GEMINI.md:100-105
Integration Steps
1. Domain Service Registration
New providers must be added to the LLMCascade domain service. According to the Hex-Lite + DDD pattern, these services reside in worker/src/services/. These services are responsible for the logic of the reasoning cascade and prompt building.
Sources: GEMINI.md:166-175
2. Prompt Configuration Updates
The system uses the Ultimate Content Intelligence & Implementation System (UCIS) framework. Version 5.1 includes specific monetization and commercial yield dimensions that new providers must be capable of parsing.
- System Prompt: Update the system prompt (e.g.,
UCIS_V5_1_SYSTEM) to ensure the new model follows the "Closed Universe" and "Transcript Absolutism" directives. - JSON Schema: Ensure the provider can output a valid JSON object conforming to the v2.0 schema, including dimensions, knowledge graphs, and monetization verdicts.
Sources: web/lib/prompts/ucis-v5.1.ts:1-20, supabase/migrations/20260611142500_add_prompt_config.sql:535-585
3. Verification & Validation
All new providers must pass the structural and content rigor checks defined in the UCISValidator.
| Requirement | Description | Source Reference |
|---|---|---|
| Persona Header | Must include Primary, Secondary, and Tertiary persona weights. | ucis-v5.1.ts:77-88 |
| 11 Dimensions | Must support all 11 intelligence dimensions or invoke the "Insufficient Data Protocol". | ucis-v5.1.ts:51-60 |
| JSON v2.0 | Output must be a single, valid JSON envelope. | ucis-v5.1.ts:474-515 |
| Zero Inference | Provider is forbidden from hallucinating data outside the transcript. | ucis-v5.1.ts:40-48 |
Sources: web/lib/prompts/ucis-v5.1.ts:77-100, web/lib/tests/ucis-v5-validator.test.ts:1-50
Infrastructure Requirements
Cloudflare Worker Streaming
Analytical route handlers for new providers must implement dynamic response streaming (SSE). This is critical to maintain the HTTP connection beyond standard serverless limits.
// Example from Architectural Laws
const response = new NextResponse();
response.headers.set('Content-Type', 'application/json');
// Chunk data back to client as it generates to extend window to ~25-90s
Sources: CLAUDE.md:21-23, GEMINI.md:107-115
S2S Persistence
Once an analysis is completed by a provider, the results are saved back to Supabase via the /api/analyses/persist endpoint. This requires a ContentSignature using HMAC to ensure the data has not been tampered with during the transit from the LLM provider to the worker.
Sources: README.md:26-27, conductor/tracks.md:40-45
Conclusion
Adding a new LLM provider to hex-yt-intel requires positioning the model within the existing LLM Cascade, ensuring compatibility with the UCIS v5.1 JSON output schema, and implementing the Hybrid Edge Symphony streaming protocols. This ensures the engine maintains its 100% reliability and zero-timeout standard across diverse content lengths.