Technology Stack
Relevant source files
The following files were used as context for generating this wiki page:
The "Analisis Sentiment Data Mining" project employs a modern, dual-architecture technology stack designed to provide both historical analysis and real-time insights into YouTube comments, with a focus on Indonesian football. The system is split into a Python-based backend for AI/ML processing and a Next.js frontend for interactive data visualization.
A key feature is its Dual Model Architecture. For static, historical data, it uses a custom-trained Support Vector Machine (SVM) model, which is fast, cost-effective, and consistent. For live, unprocessed data, it leverages the Google Gemini AI, which offers superior contextual understanding, reasoning, and handling of Indonesian slang. This hybrid approach allows the system to use the best tool for each specific task, balancing speed, cost, and accuracy.
Sources: README.md:10-23, CHANGELOG.md:7-20
Architecture Overview
The project is logically separated into a frontend application, a backend API server, and external services. The frontend is a Next.js application responsible for all user interactions and visualizations. The backend is a FastAPI server that exposes endpoints for sentiment analysis, data fetching, and AI model interaction.
This diagram illustrates the high-level system architecture.
Frontend (dashboard-next)
The frontend is a modern web application built with Next.js 14, utilizing the App Router for code splitting and server-side rendering. It is written in TypeScript for type safety and improved developer experience.
Sources: dashboard-next/README.md:13-19
Core Technologies
| Technology | Purpose |
|---|---|
| Next.js 14 | React framework for server-side rendering, static site generation, and routing. |
| TypeScript | Provides static typing for JavaScript, enhancing code quality and maintainability. |
| Tailwind CSS | A utility-first CSS framework for rapid UI development and consistent styling. |
| Framer Motion | A production-ready motion library for creating smooth animations and interactions. |
| Recharts | A composable charting library for building interactive data visualizations. |
| Lucide React | A library of modern, clean icons. |
Sources: dashboard-next/README.md:13-19, PERFORMANCE_OPTIMIZATION_REPORT.md:58-69
Performance Optimizations
The frontend incorporates several performance optimizations to ensure a fast and responsive user experience:
- Server-Side Caching: API routes use in-memory caching to reduce redundant computations.
- Client-Side Caching: SWR is configured with a 5-minute deduplication interval to minimize network requests.
- Code Splitting: Heavy components like charts and AI insights are lazy-loaded using
next/dynamic. - Memoization:
React.memoanduseMemoare used to prevent unnecessary component re-renders. - Build Optimization: The Next.js configuration enables SWC minification, Gzip/Brotli compression, and tree-shaking for libraries like
recharts.
Sources: PERFORMANCE_OPTIMIZATION_REPORT.md:18-72, dashboard-next/PERFORMANCE_OPTIMIZATIONS.md
Backend (Python API)
The backend is a high-performance API server built with FastAPI. It handles all data processing, machine learning inference, and communication with external AI services.
Sources: README.md:25-32
Core Technologies
| Technology | Purpose |
|---|---|
| FastAPI | A modern, fast web framework for building APIs with Python, with automatic interactive documentation. |
| Google Gemini AI | Used for real-time, context-aware sentiment analysis and smart comment filtering. |
| Scikit-learn | Provides the SVM model implementation and TF-IDF vectorization for static analysis. |
| Sastrawi | A library for Indonesian language stemming, used in text preprocessing. |
| Joblib | Used for saving and loading the pre-trained SVM model. |
| Pandas | (Inferred) Used for data manipulation and reading CSV files on the backend. |
Sources: README.md:34-41, dashboard-next/app/model/page.tsx:132-135, dashboard-next/app/settings/page.tsx:64-65
Dual Model System
The project's core innovation is its dual-model architecture, which selects the appropriate analysis model based on the use case.
This diagram shows the logic for how the system chooses between the SVM and Gemini AI models.
Sources: README.md:10-23, CHANGELOG.md:7-20
Model Comparison
| Feature | SVM Model (Static) | Gemini AI (Live) |
|---|---|---|
| Data Source | Pre-labeled & cleaned CSV | Real-time & raw YouTube comments |
| Accuracy | 87.5% (domain-specific) | 95%+ (general context) |
| Speed | <1ms per comment | 1-2s per comment |
| Context | Limited | Excellent |
| Cost | Free (offline) | API quota-based |
| Reasoning | No | Yes |
Sources: README.md:25-32
Data and API Layer
The system sources data from two primary locations: a static CSV file for historical analysis and the YouTube Data API for live comments. These are processed by the backend and exposed to the frontend via a set of internal and external APIs.
Data Flow
The data flows from its source, through the backend for processing, and is finally consumed by the frontend dashboard.
API Interaction Sequence
This sequence diagram shows a typical interaction for fetching and analyzing live comments.
Conclusion
The technology stack for the "Analisis Sentiment Data Mining" project is a well-architected system that leverages modern frameworks and a strategic dual-model AI approach. By combining the speed of a pre-trained SVM model for historical data with the contextual power of Gemini AI for live analysis, the project delivers a robust, accurate, and highly performant platform. The separation of concerns between the FastAPI backend and the Next.js frontend, coupled with extensive performance optimizations, results in a scalable and maintainable solution for advanced sentiment analysis.