Monitoring and Drift Detection

Relevant source files

The following files were used as context for generating this wiki page:

• PERFORMANCE_OPTIMIZATION_REPORT.md
• DEPLOYMENT_SUMMARY.md
• dashboard-next/DEPLOYMENT_CHECKLIST.md
• dashboard-next/app/api/live-comments/route.ts
• dashboard-next/app/api/gemini-sentiment/route.ts
• UNIFIED_SCORING_SYSTEM.md
• CHANGELOG.md
• README.md

Monitoring and drift detection in the Garuda Sentiment Analysis project is a comprehensive strategy to ensure system reliability, performance, and consistent analysis quality. It encompasses post-deployment checks, continuous performance tracking, and robust fallback mechanisms to handle service degradation or failure. While "drift detection" is not implemented in a formal machine learning sense (i.e., tracking statistical changes in data distribution), the project addresses behavioral drift by managing a dual-model architecture and implementing a fallback system for when primary services like the Gemini AI API are unavailable.

This system ensures high uptime, a smooth user experience, and predictable analysis results by defining clear success criteria and providing tools and procedures for tracking key metrics. The monitoring strategy covers server-side APIs, client-side application performance, and overall user experience.

Sources: PERFORMANCE_OPTIMIZATION_REPORT.md, DEPLOYMENT_CHECKLIST.md, README.md

Post-Deployment Monitoring Strategy

After each deployment, a structured monitoring process is followed to quickly identify and address any issues. This process is divided into time-based phases, from immediate checks to long-term observation.

Sources: dashboard-next/DEPLOYMENT_CHECKLIST.md:73-93

Monitoring Phases

The monitoring process is broken down into three key timeframes:

• Immediate Checks (First 5 minutes): Focuses on critical path functionality to ensure the deployment was successful at a basic level.
• Performance Monitoring (First hour): Verifies that the system is stable and performant under normal load.
• User Experience (First day): Assesses the impact on user behavior and satisfaction.

The following diagram illustrates the post-deployment monitoring timeline and checks.

Key Monitoring Metrics

The project defines specific metrics across server, client, and user experience categories to track system health and performance. These metrics are aligned with predefined success criteria to provide clear benchmarks for evaluation.

Sources: PERFORMANCE_OPTIMIZATION_REPORT.md:200-221, dashboard-next/DEPLOYMENT_CHECKLIST.md:109-123

Server Metrics

Metric	Description	Recommended Tool
API Response Times	Time taken for API endpoints to respond.	Vercel Analytics
Cache Hit Rate	Percentage of requests served from cache.	Vercel Analytics
Memory Usage	Server memory consumption.	Vercel Analytics
CPU Usage	Server CPU utilization.	Vercel Analytics
Error Rate	Percentage of requests resulting in an error.	Sentry / Vercel

Sources: PERFORMANCE_OPTIMIZATION_REPORT.md:202-207

Client & User Experience Metrics

Metric	Target	Description	Recommended Tool
Core Web Vitals
LCP (Largest Contentful Paint)	< 2.5s	Time to render the largest content element.	Lighthouse
FID (First Input Delay)	< 100ms	Time from first user interaction to response.	Lighthouse
CLS (Cumulative Layout Shift)	< 0.1	Visual stability of the page.	Lighthouse
TTFB (Time to First Byte)	< 600ms	Time for the browser to receive the first byte.	Lighthouse
User Engagement
Bounce Rate	Varies	Percentage of single-page sessions.	Vercel Analytics
Session Duration	Varies	Average length of a user session.	Vercel Analytics

Sources: PERFORMANCE_OPTIMIZATION_REPORT.md:89-95, 209-216, dashboard-next/DEPLOYMENT_CHECKLIST.md:110-112

Drift and Failure Handling

The system includes a robust fallback mechanism within its API routes to handle failures or "drift" in service availability, particularly for external dependencies like the Gemini AI API. This ensures that the application remains functional, albeit with potentially different behavior, rather than failing completely.

Sources: dashboard-next/app/api/live-comments/route.ts, dashboard-next/app/api/gemini-sentiment/route.ts

API Fallback System

The live-comments and gemini-sentiment API routes are designed to first call the primary backend service (Unified Scorer at http://localhost:8000). If this service is unavailable or fails, the API "drifts" to a built-in fallback sentiment analyzer.

This fallback function uses a predefined lexicon and a weighted scoring algorithm to perform sentiment analysis locally, ensuring the application continues to provide sentiment data.

The sequence diagram below shows the logic for an API call to /api/gemini-sentiment.

Fallback Analyzer Logic

The fallback analyzer is a self-contained function within the API route that provides a consistent, rule-based sentiment score.

1. Lexicon-Based Scoring: It uses a lexicon of positive and negative words with 'strong', 'medium', and 'weak' weights.
2. Weighted Calculation: It calculates a total positive and negative score based on the words found in the text.
3. Unified Thresholds: It applies a ratio-based logic to classify sentiment as 'positive', 'negative', or 'neutral', ensuring its behavior is predictable.

// Source: dashboard-next/app/api/live-comments/route.ts:5-55
function analyzeSentimentFallback(text: string) {
  // Unified lexicon for consistent fallback scoring
  const sentimentLexicon = {
    positive: {
      strong: ['sangat bagus', 'luar biasa', 'excellent', /* ... */],
      medium: ['bagus', 'mantap', 'keren', /* ... */],
      weak: ['lumayan', 'cukup', 'ok', /* ... */]
    },
    negative: {
      strong: ['sangat jelek', 'sangat buruk', 'terrible', /* ... */],
      medium: ['jelek', 'buruk', 'kecewa', /* ... */],
      weak: ['kurang', 'tidak', 'bukan', /* ... */]
    }
  };
  
  // ... scoring logic ...
  
  const posRatio = positiveScore / totalScore;
  const negRatio = negativeScore / totalScore;
  
  // Unified decision thresholds
  if (posRatio >= 0.35 && posRatio > negRatio) {
    return { sentiment: 'positive', /* ... */ };
  } else if (negRatio >= 0.35 && negRatio > posRatio) {
    return { sentiment: 'negative', /* ... */ };
  } else {
    return { sentiment: 'neutral', confidence: 0.65 };
  }
}

This fallback mechanism is a critical part of monitoring and resilience, preventing catastrophic failure and ensuring service continuity.

Sources: dashboard-next/app/api/live-comments/route.ts:5-55, dashboard-next/app/api/gemini-sentiment/route.ts:38-118

Model Behavior Monitoring

The project utilizes a dual-model architecture (SVM for static data, Gemini AI for live data) and a Unified Scoring System. This system allows for different scoring behaviors (static, realtime, unified) to be applied based on the context. Monitoring model behavior involves ensuring that the correct scoring profile is used and that the results are consistent. The Unified Scoring System centralizes the logic, making it easier to track and manage how different models behave.

Sources: UNIFIED_SCORING_SYSTEM.md, CHANGELOG.md:5-18, README.md:16-36

Recommended Tools

The project documentation recommends a set of tools for performance auditing, profiling, and error tracking.

Tool	Purpose
Lighthouse	Performance audits
Chrome DevTools	Network/Perf profiling
Vercel Analytics	Real user monitoring
Sentry	Error tracking
npx autocannon	Load testing

Sources: PERFORMANCE_OPTIMIZATION_REPORT.md:218-223, dashboard-next/DEPLOYMENT_CHECKLIST.md:102-107

Summary

The monitoring and drift detection strategy for the Garuda project is a multi-faceted approach focused on maintaining high performance and reliability. It combines proactive post-deployment checks, continuous tracking of key server and client metrics against defined success criteria, and a robust API fallback system to handle service degradation gracefully. This ensures that developers can quickly identify issues, measure the impact of changes, and provide a stable and consistent experience for users.