Cloudflare Streaming Worker
Relevant source files
The following files were used as context for generating this wiki page:
The Cloudflare Streaming Worker is the high-performance execution engine of the hex-yt-intel project, specifically engineered to handle high-latency LLM (Large Language Model) synthesis and real-time metadata extraction. It forms the second pillar of the Hybrid Edge Symphony (v1.4.1), enabling the system to bypass the execution limits of standard serverless environments like Vercel. While the frontend and authentication are managed on Vercel, the Cloudflare Worker maintains long-lived Server-Sent Events (SSE) connections for deep content analysis, typically lasting up to 58 seconds or more.
Sources: README.md:16-24, CLAUDE.md:17-21, GEMINI.md:144-150
Architecture: The Hybrid Edge Symphony
The worker operates within a stratified multi-cloud architecture. It is designed as a "High-Performance Streaming Engine" using the Hono.js framework. Its primary role is to execute the Ultimate Content Intelligence System (UCIS) v5.1 prompts against a cascade of LLMs, providing a direct browser-to-edge connection for streaming results.
Sources: README.md:27-32, worker/package.json:20, skill/README.md:85-91
Data Flow and Lifecycle
The request lifecycle follows a specific sequence to ensure security and reliability:
- Vercel Bouncer: Validates high-security authentication and atomic quotas.
- Client Transition: The client receives a signed token and initiates an SSE request to the Cloudflare Worker.
- Cloudflare Streamer: The worker initiates LLM synthesis and streams chunks of markdown or JSON back to the client.
- HMAC Persistence: Upon completion, the worker performs a cryptographically signed server-to-server (S2S) write to the database (Supabase) to guarantee data integrity.
Sources: README.md:16-24, CLAUDE.md:17-21, GEMINI.md:111-118
System Flow Diagram
The following diagram illustrates the interaction between the Vercel infrastructure, the Cloudflare Worker, and external services.
The diagram shows the transition from the Vercel authentication gate to the Cloudflare streaming execution environment.
Sources: README.md:16-24, CLAUDE.md:17-21, skill/README.md:95-103
Core Functionality
1. Metadata Extraction
The worker serves as a real-time metadata fetcher, querying the YouTube Data API v3. It returns live video details including titles, descriptions, view counts, and engagement metrics (likes, comments). This data is used to populate the analysis frameworks.
Sources: skill/README.md:85-91, skill/README.md:169-183
2. LLM Cascade and Synthesis
The worker implements an LLM Model Cascade (ADR 003). It utilizes a sequence of models, starting with efficient leads (e.g., Nemotron-3-Nano) and falling back to higher-capacity models (e.g., Haiku 4.5) if necessary. This ensures analysis completion even during API instability.
Sources: CLAUDE.md:43-43, GEMINI.md:140-142
3. Response Execution Protocols
The worker adheres to strict "Backend Laws" to maintain stability:
- Stratified Dual-Timeouts: Implements a 3-second connection handshake timeout and a 90-second maximum read window for token streaming.
- Dynamic Streaming: Responses are chunked and streamed instantly to the client to extend the connection lifetime beyond standard 10-second limits.
Sources: CLAUDE.md:11-15, GEMINI.md:111-125
API Reference
Endpoints
| Endpoint | Method | Description |
|---|---|---|
/fetch-metadata | GET | Retrieves live YouTube metadata using a video_id. |
/analyze-llm-stream | POST | Initiates the SSE streaming synthesis using the UCIS framework. |
Sources: skill/README.md:85-91, web/lib/types/contracts.ts:100-105
Request Contract: WorkerStreamRequest
To initiate synthesis, the worker expects a WorkerStreamRequest payload, typically signed with an HMAC signature (sig) and an expiration timestamp (exp).
| Field | Type | Description |
|---|---|---|
videoId | string | The 11-character YouTube video ID. |
analysisId | string | Unique identifier for the analysis record. |
transcript | string | The raw text transcript of the video. |
persona | string | Target persona (e.g., p1 for Content Creator). |
sig | string | HMAC signature for request verification. |
models | string[] | Optional per-tier model cascade configuration. |
Sources: web/lib/types/contracts.ts:100-115
Technical Stack & Configuration
The worker is built with modern edge-compatible technologies to ensure minimal latency and high reliability.
- Framework: Hono.js (v4.12.25) for high-performance routing.
- Validation: Zod (v4.4.3) for strict schema enforcement of inbound payloads.
- Observability: Integrated with Sentry for edge error tracking and performance monitoring.
- Runtime: Cloudflare Workers (Edge Runtime).
Sources: worker/package.json:18-24, web/app/admin/dashboards/page.tsx:156-160
Configuration (wrangler.toml)
Key environment variables and secrets required for worker operation include:
YOUTUBE_API_KEY: Secret used for authenticating with the YouTube Data API.HMAC_SECRET: Used to verify signatures from the Vercel Bouncer.OPENROUTER_API_KEY: Credentials for the LLM cascade providers.
Sources: skill/README.md:90-90, CLAUDE.md:55-55
Conclusion
The Cloudflare Streaming Worker is critical to the hex-yt-intel infrastructure, providing the raw computational power and connection stability required for deep intelligence synthesis. By decoupling high-latency LLM tasks from the Vercel frontend, the worker ensures 100% reliability for complex UCIS v5.1 analyses, even for long-form content exceeding 90 minutes.
Sources: README.md:20-24, skill/src/prompts.ts:15-20