Live Data Components
Relevant source files
The following files were used as context for generating this wiki page:
The Live Data Components provide real-time sentiment analysis of YouTube comments. This system is designed for live monitoring and social listening, featuring a dual-model architecture that leverages Google's Gemini AI for primary analysis and a robust local fallback. Key features include fetching live comments from YouTube, applying a smart filter to isolate relevant content, and performing aggressive sentiment analysis to provide clear, actionable insights with minimal neutral results.
The main user interface for this system is the Live Sentiment Analysis page (/realtime), which allows users to start and stop monitoring streams for specific YouTube videos or automatically search for the latest videos on a given topic. The system is a core part of the v5.0 release, which introduced this advanced AI-powered, real-time functionality.
Sources: README.md, dashboard-next/app/realtime/page.tsx, CHANGELOG.md
Architecture Overview
The live data system is composed of a Next.js frontend, a set of backend API routes, and external services including the YouTube Data API and a separate FastAPI server running the Gemini AI model. The frontend initiates requests, the API routes orchestrate data fetching and processing, and the external services provide the raw data and advanced AI analysis.
This diagram illustrates the high-level architecture of the live data components.
Frontend Components
Live Sentiment Analysis Page (/realtime)
This is the central hub for controlling and viewing live sentiment analysis. It allows users to input a YouTube URL for monitoring or leave it blank to auto-search for recent videos related to "timnas indonesia".
Sources: dashboard-next/app/realtime/page.tsx
Key Features
- Stream Control: Users can start and stop the data-fetching stream. The UI provides visual feedback on the stream's status (LIVE/OFFLINE).
- URL Input: An input field to specify a target YouTube video URL.
- Auto-Search: If no URL is provided, the system automatically searches for and selects the latest relevant video.
- Real-time Statistics: Displays key metrics like Total Comments, Positive, Negative, and Users, which are updated periodically.
- Visualizations: Includes a
PieChartfor sentiment distribution and aBarChartfor live emotions. Note that the emotion data is sample data augmented with live stats, not a direct result of emotion analysis on comments. - Recent Comments Feed: Shows the most recent comments processed by the Gemini AI, including the sentiment, confidence, and the AI's reasoning.
State Management
The page uses React hooks (useState, useEffect) to manage its state:
isStreaming: Tracks the live status of the stream, controlled via a sharedStreamContext.videoUrl: Stores the user-provided YouTube URL.currentVideo: Holds metadata of the video being monitored.liveStats: Contains aggregated sentiment counts (total, positive, negative, neutral).recentComments: An array of the latest comments with their analysis results.
Sources: dashboard-next/app/realtime/page.tsx:9-20
Data Fetching
handleStartStream(): Initiates the stream. It either callsfetchRealCommentswith a user-provided URL orautoSearchVideosto find a video automatically.fetchRealComments(url): Fetches comments and AI analysis from the/api/youtubeendpoint (which appears to be a proxy or wrapper for/api/live-comments).useEffectHook: WhenisStreamingis true, this hook sets up an interval to refresh the comment data every 30 seconds.
Sources: dashboard-next/app/realtime/page.tsx:68-103
LiveComments Component
This component is responsible for displaying a live feed of comments. It receives the streaming status as a prop and likely fetches and displays comments internally.
Sources: dashboard-next/components/LiveComments.tsx, dashboard-next/app/realtime/page.tsx:231
Backend API
/api/live-comments
This is the core API endpoint that orchestrates fetching, filtering, and analyzing YouTube comments. It is a GET endpoint that accepts several query parameters to control its behavior.
Sources: dashboard-next/app/api/live-comments/route.ts
Request Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
videoId | string | lDtSjKb_8Jo | The ID of the YouTube video to fetch comments from. |
sentiment | boolean | false | If true, performs sentiment analysis on the comments. |
filter | boolean | true | If true, applies the Smart Comment Filter to the fetched comments. |
maxResults | number | 200 | The maximum number of comments to return. |
Sources: dashboard-next/app/api/live-comments/route.ts:200-204
Data Flow and Processing
The following diagram shows the sequence of operations within the /api/live-comments endpoint.
Smart Comment Filtering
To ensure the quality and relevance of the analyzed comments, a sophisticated filtering system is applied. The filterRelevantComments function uses a scoring system to decide whether a comment is relevant.
Sources: dashboard-next/app/api/live-comments/route.ts:79-152
Filtering Criteria
| Category | Description | Score Impact |
|---|---|---|
| Context | Contains football-related keywords (e.g., timnas, pemain, pssi, sty). | +3 |
| Opinion | Contains opinion words (e.g., bagus, jelek, harusnya). | +2 |
| Length | Comment is longer than 30 characters. | +1 |
| Question | Contains a question mark ?. | +1 |
| Spam | Contains spam keywords (e.g., subscribe, promo, http). | -5 |
| Quality | Too short (< 8 chars), mostly emojis, excessive repeated characters, or all caps. | -1 to -3 |
A comment is considered relevant if its final relevanceScore is greater than 0. This system effectively removes spam and low-quality content, focusing the AI analysis on meaningful opinions.
Sources: dashboard-next/app/api/live-comments/route.ts:123-146
Sentiment Analysis
The system uses a dual-model approach for sentiment analysis, prioritizing a powerful external AI model but including a reliable local fallback.
Sources: dashboard-next/app/api/live-comments/route.ts:155-196, UNIFIED_SCORING_SYSTEM.md
1. Primary: Unified Scorer (Gemini AI)
The analyzeSentimentWithUnified function makes a POST request to the FastAPI server at http://localhost:8000/predict. It sends the comment text and specifies model_type: "realtime" to use an "aggressive" scoring model optimized for live analysis, which aims to reduce neutral classifications. If successful, it returns a detailed analysis including sentiment, confidence, and AI reasoning.
Sources: dashboard-next/app/api/live-comments/route.ts:157-180, UNIFIED_SCORING_SYSTEM.md
2. Secondary: Fallback Analyzer
If the Gemini AI API call fails or times out (after 10 seconds), the system gracefully degrades to the analyzeSentimentFallback function. This local function uses a weighted lexicon of positive and negative words to calculate a sentiment score, ensuring the system remains operational. The lexicon and scoring logic are part of the Unified Scoring System, ensuring results are as consistent as possible with the primary model.
Sources: dashboard-next/app/api/live-comments/route.ts:6-77, dashboard-next/app/api/live-comments/route.ts:189-195