Observability & Monitoring
Relevant source files
The following files were used as context for generating this wiki page:
The Observability & Monitoring system in hex-yt-intel provides high-fidelity tracking of system health, performance metrics, and application errors. It utilizes a multi-layered approach involving Sentry for error tracking and performance profiling, custom health check endpoints, and administrative dashboards to ensure the reliability of the "Hybrid Edge Symphony" architecture.
The system is designed to handle the complexities of a distributed environment involving Vercel (Next.js), Cloudflare Workers, and Supabase. It prioritizes low-cost queries and data integrity through cached hits and cryptographically signed server-to-server writes.
Sources: README.md:16-30, GEMINI.md:120-135, web/lib/monitoring/README.md:3-8
System Architecture & Data Flow
The monitoring architecture follows a "Triple-Redundant Hybrid Model" where different components handle specific observability concerns. The flow involves instrumentation of API routes, database queries, and external service calls to provide a comprehensive view of the system's state.
The diagram above illustrates the relationship between the application, the monitoring utilities, and the external observability platforms.
Sources: web/app/admin/dashboards/page.tsx:88-150, web/lib/monitoring/README.md:10-60, scripts/test-observability.sh:44-110
Key Monitoring Components
Sentry Integration
Sentry serves as the primary engine for error tracking and performance monitoring. It is configured to capture transactions, session replays, and specific application events.
| Component | Description | Source File |
|---|---|---|
trackAPIRequest | Automatically times API requests and logs them to Sentry. | sentry-utils.ts |
trackDatabaseQuery | Wraps Supabase calls to monitor latency and query success. | sentry-utils.ts |
trackExternalCall | Monitors calls to Cloudflare Workers or OpenRouter. | sentry-utils.ts |
setUserContext | Associates errors with specific user IDs and subscription tiers. | sentry-utils.ts |
addBreadcrumb | Logs sequential steps to aid in debugging failed processes. | sentry-utils.ts |
Sources: web/lib/monitoring/README.md:10-75, scripts/test-observability.sh:78-95
Health Check System
The /api/health endpoint provides real-time status updates for critical infrastructure components. This data is consumed by the Admin Dashboard to alert operators of degradation.
The health check sequence verifies the connectivity and latency of the database and external workers.
Sources: web/app/admin/dashboards/page.tsx:10-25, scripts/test-observability.sh:44-65
Observability Dashboards
The administrative interface provides a centralized view of system health and performance metrics. It includes specific cards for tracking total analyses, active users, and API latency.
Metric Categories
- System Health: Real-time status of Database, Cloudflare Worker, and Sentry configuration.
- Key Metrics: Aggregate counts of analyses, searches, and user distribution (Pro vs. Free).
- Performance: Average API latency, 24-hour error rates, and system uptime.
Sources: web/app/admin/dashboards/page.tsx:35-85
Business Metrics Tracking
Specific business-related events are tracked to monitor product usage and cost efficiency.
// Example of tracking analysis creation with latency
const latency = performance.now() - startTime;
trackAnalysisCreated(userId, userTier, latency);
// Tracking embedding costs and token usage
trackEmbedding(userId, analysisId, tokensUsed, costUsd);
Sources: web/lib/monitoring/README.md:46-56, web/lib/monitoring/README.md:131-140
Implementation Laws & Fail-safes
The monitoring system implements specific "Backend Laws" to ensure performance and cost control:
- Pre-Query Cache Hit Circuit: Before processing a new analysis, the system queries the
analysestable. If a match is found, cached markdown is returned instantly, reducing token costs to $0. - Stratified Dual-Timeouts:
- Connection Handshake: 3-second hard timeout to detect network faults.
- Token Streaming Window: 25-second maximum read to stay within Vercel execution limits.
- Dynamic Response Streaming: Route handlers implement chunked response streaming to extend connection lifetimes beyond standard 10-second limits.
Sources: GEMINI.md:120-145, README.md:18-25
Testing & Verification
A dedicated test suite (test-observability.sh) is used to verify the monitoring infrastructure.
| Test Phase | Verified Elements |
|---|---|
| Component 1 | /api/health returns valid JSON with component statuses. |
| Component 2 | Sentry DSN, tracing, and session replays are configured. |
| Component 3 | Presence of monitoring utilities (sentry-utils.ts, metrics.ts). |
| Component 7 | System instrumentation via instrumentation.ts. |
Sources: scripts/test-observability.sh:10-140
Summary
Observability & Monitoring in hex-yt-intel is a mission-critical subsystem that enables the "Hybrid Edge Symphony." By combining deep Sentry integration with custom health checks and strict timeout laws, the project ensures that high-latency LLM synthesis is reliable and cost-effective. The system provides developers and admins with the necessary tools to track everything from low-level database latency to high-level business metrics like embedding costs and user conversion.
Sources: README.md:16-30, web/lib/monitoring/README.md:160-170