Home
Relevant source files
The following files were used as context for generating this wiki page:This project is an advanced, real-time AI sentiment analysis system designed specifically for YouTube comments related to Indonesian football. It leverages a sophisticated dual-model architecture to provide both historical analysis on cleaned data and live, context-aware analysis on raw, real-time comments. The system is built with a FastAPI backend for AI/ML processing and a Next.js 14 frontend for interactive data visualization.
Key features include an "Aggressive AI Sentiment Analysis" model to minimize ambiguous neutral results, a "Smart Comment Filter" powered by Gemini AI to isolate relevant discussions, and a highly optimized, responsive dashboard. The project is engineered for performance, with multi-layer caching and a design philosophy of "progressive disclosure" to ensure a smooth user experience.
Sources: README.md, dashboard-next/README.md, PERFORMANCE_OPTIMIZATION_REPORT.md
System Architecture
The core of the project is its dual-model architecture, which uses the best model for each specific task: a pre-trained SVM model for static, historical data and Google's Gemini AI for live, contextual analysis.
Sources: README.md:10-23, CHANGELOG.md:13-22
This diagram shows the two distinct data paths: static pages use the SVM model on pre-processed data, while live pages use the Smart Filter and Gemini AI on raw YouTube comments.
Model Comparison
The two models are optimized for different use cases, providing a balance between speed, cost, and contextual understanding.
| Feature | SVM Model | Gemini AI |
|---|---|---|
| Data Source | Pre-labeled & cleaned CSV | Real-time & raw YouTube comments |
| Accuracy | 87.5% (domain-specific) | 95%+ (general context) |
| Speed | <1ms per comment | 1-2s per comment |
| Context | Limited | Excellent |
| Cost | Free (offline capable) | API quota applies |
| Reasoning | No | Yes |
Sources: README.md:25-34, dashboard-next/README.md:8-21
Dashboard Structure
The Next.js dashboard is organized into logical sections to provide a clean user experience based on the principle of "Progressive Disclosure". Users start with a high-level overview and can drill down into more detailed analyses.
Sources: dashboard-next/DASHBOARD_STRUCTURE.md, dashboard-next/MENU_GUIDE.md
This diagram illustrates the main navigation structure, separating core analysis pages from utility tools.
User Journey Philosophy
The structure is designed to cater to different user personas:
- Casual User: Views the main
Dashboardfor a quick snapshot. - Analyst: Navigates from
DashboardtoSentimentandEmotionsfor deeper insights. - Researcher: Uses all pages, with a focus on the
Comments ExplorerandModel Performance. - Developer: Focuses on
Model Performance,Documentation, and theLive Analyzer.
Sources: dashboard-next/DASHBOARD_STRUCTURE.md:73-79
Core Features
Smart Comment Filtering
To ensure high-quality analysis, the system employs an AI-powered Smart Comment Filter for live data. This filter processes raw comments to remove spam, promotions, and irrelevant content, isolating comments that are genuinely related to football.
The filtering logic uses a scoring system that rewards comments with football context and opinionated words, while penalizing spam, short comments, and emoji-only posts.
Sources: README.md:82-89, dashboard-next/app/api/live-comments/route.ts:70-143
This flow shows the decision process within the filterRelevantComments function to determine if a comment is relevant.
Unified Sentiment Scoring
To maintain consistency between the different models, a "Unified Scoring System" is implemented. This system uses a shared sentiment lexicon and decision thresholds but applies model-specific adjustments.
- Static Mode: Mimics the original SVM's conservative behavior, favoring neutral classifications.
- Realtime Mode: Uses an "aggressive" configuration that boosts clear positive/negative signals and reduces neutral results, ideal for live analysis.
This ensures that while each model is optimized for its task, the underlying scoring logic remains predictable and easy to maintain.
Sources: UNIFIED_SCORING_SYSTEM.md, dashboard-next/app/api/live-comments/route.ts:6-68
Data Flow & APIs
Static Data Flow (/api/stats)
The main dashboard pages are populated by the /api/stats endpoint, which reads from a pre-processed CSV file. This endpoint calculates all necessary statistics, such as sentiment distribution, top emotions, and target criticism counts.
To optimize performance, the results are held in an in-memory cache for 30 seconds, and HTTP Cache-Control headers are set to allow for client-side and CDN caching.
Sources: dashboard-next/app/api/stats/route.ts, PERFORMANCE_OPTIMIZATION_REPORT.md:20-31, create_correct_sentiment_data.py
This diagram shows the data flow for loading the main dashboard, including the server-side caching mechanism.
Live Data Flow (/api/live-comments)
The live analysis pages use the /api/live-comments endpoint. This endpoint fetches recent comments from the YouTube Data API, passes them through the Smart Comment Filter, and then analyzes the sentiment of the relevant comments using the Unified Scorer in "realtime" mode.
Sources: dashboard-next/app/api/live-comments/route.ts, README.md:100-112
This diagram illustrates the process of fetching, filtering, and analyzing live YouTube comments.
Performance & Optimization
The project has undergone significant performance optimization to ensure a fast and responsive user experience, even with a large dataset.
| Metric | Before | After | Improvement |
|---|---|---|---|
| API Response (cached) | ~200ms | <5ms | 🚀 97.5% faster |
| Network Requests | High | -90% | 📉 90% reduction |
| Bundle Size | ~450KB | ~300KB | 📦 33% smaller |
| Startup Time | ~500ms | 172ms | ⚡ 65% faster |
Sources: PERFORMANCE_OPTIMIZATION_REPORT.md:88-95
Key optimizations include:
- Server-Side Caching: In-memory caching on API routes to avoid re-processing the 19,228-row CSV file on every request.
- Client-Side Caching: Optimized SWR configuration to reduce network requests and prevent UI flashing.
- React Memoization: Prevents unnecessary re-renders of components, leading to a smoother UI.
- Code Splitting: Heavy components like charts and AI insights are lazy-loaded to reduce the initial bundle size and improve page load times.
- Build Configuration: The Next.js build is optimized with SWC minification and package import optimization for smaller, faster builds.
Sources: PERFORMANCE_OPTIMIZATION_REPORT.md:17-85
Conclusion
The Garuda Sentiment Analysis project is a comprehensive and high-performance platform for analyzing public opinion on Indonesian football. Its innovative dual-model architecture provides the flexibility to perform both rapid historical analysis and nuanced, real-time contextual evaluation. With a strong focus on data quality through smart filtering and a highly optimized, user-centric dashboard, the system delivers meaningful and actionable insights from vast amounts of unstructured social media data.