Cryptographic Data Persistence
Relevant source files
The following files were used as context for generating this wiki page:
Cryptographic Data Persistence is a core security system within the Hybrid Edge Symphony architecture of the hex-yt-intel project. Its primary purpose is to guarantee data integrity during high-latency LLM synthesis by utilizing cryptographically signed server-to-server (S2S) communications. This mechanism ensures that content generated at the edge (Cloudflare Workers) is securely persisted to the primary database (Supabase) without exposing sensitive database keys to the client or risking unauthorized data manipulation.
Sources: README.md:21-25, CLAUDE.md:14-18, GEMINI.md:124-126
Architectural Overview
The persistence logic follows the Hexagonal Lite (Ports & Adapters) pattern. It decouples the business requirement of persistence from the underlying cryptographic implementation, allowing for secure, tamper-proof writes across different cloud environments.
The Hybrid Flow
The system operates across a multi-cloud environment to bypass standard serverless execution limits:
- Vercel (Bouncer): Handles authentication and quota enforcement.
- Cloudflare (Streamer): Conducts the high-latency LLM synthesis.
- HMAC Persistence: Signs and verifies the resulting analysis before saving it to the database.
Sources: README.md:21-25, GEMINI.md:162-167, AGENTS.md:20-25
Data Flow Diagram
The following diagram illustrates the lifecycle of an analysis from request to cryptographic persistence.
This diagram shows the transition from the Vercel-based authentication gate to the Cloudflare-based synthesis engine, culminating in a cryptographically verified database write.
Sources: README.md:21-25, CLAUDE.md:14-18, AGENTS.md:20-25
Key Components and Logic
Persistence Port and Verification
The system defines clear interfaces for cryptographic operations. The verifyContentSig function is a critical security gate that validates incoming data from the edge worker.
| Component | Responsibility | Source File |
|---|---|---|
PersistencePort | Defines the abstract interface for saving analysis data. | GEMINI.md:162-165 |
verifyContentSig | Hardened function to verify the HMAC signature of incoming stream content. | AGENTS.md:23 |
STREAM_HMAC_SECRET | The environment variable used as the secret key for signing and verification. | AGENTS.md:21 |
Sources: AGENTS.md:21-23, GEMINI.md:162-165
API Implementation: /persist
The /web/app/api/analyses/persist/route.ts endpoint serves as the receiver for synthesized data. It is specifically hardened to prevent unauthorized access by requiring a valid HMAC signature. This ensures that only the authorized Cloudflare Worker can trigger database updates for a specific analysis record.
Sources: AGENTS.md:23, README.md:25, web/app/api/analyses/persist/route.ts
Cryptographic Identity and Tokens
The system utilizes specific utilities for managing stream tokens and cryptographic signatures:
CryptographicTokenPort.ts: An abstract interface defining how tokens are handled.stream-token.ts: Contains the implementation logic for generating and validating tokens used during the streaming lifecycle.
Sources: web/lib/ports/CryptographicTokenPort.ts, web/lib/stream-token.ts
Security Configuration
The security of the persistence layer relies on the synchronization of secrets and the enforcement of specific execution timeouts.
| Configuration | Value / Limit | Purpose |
|---|---|---|
| HMAC Secret | STREAM_HMAC_SECRET | Used for S2S tamper-proof signing. |
| Connection Timeout | 3 seconds | Prevents hanging connections during handshake. |
| Execution Window | 25s (Vercel) / 58s (CF) | Managed through response streaming to prevent timeouts. |
| Authentication Pattern | getSupabaseClientWithAuth() | Server-side only pattern for secure DB access. |
Sources: CLAUDE.md:10-18, GEMINI.md:104-106, README.md:22-24
Implementation Sequence
The implementation of this system follows a strict vertical execution flow to ensure stability:
- Secret Validation: Ensure
STREAM_HMAC_SECRETis present in the environment. - Bouncer Refactor: Configure the Next.js route as a fast token gate.
- Persistence Hardening: Implement the
verifyContentSigcheck on the persist endpoint. - Frontend Sync: Update hooks (e.g.,
useSSEStream.ts) to manage the direct connection to the Cloudflare Worker.
Sources: AGENTS.md:20-25
Conclusion
Cryptographic Data Persistence provides a robust bridge between high-performance edge computing and centralized data storage. By utilizing HMAC-verified signatures for server-to-server communication, the hex-yt-intel project achieves 100% data integrity and reliability, successfully overcoming the execution limits of standard serverless functions while maintaining a high security posture.
Sources: README.md:26-27, CLAUDE.md:14-18