Feature: Aggressive Sentiment Analysis
Relevant source files
The following files were used as context for generating this wiki page:
The Aggressive Sentiment Analysis is a specialized feature designed to provide more decisive sentiment classifications by minimizing "neutral" results. It is a key component of the project's dual-model architecture, specifically optimized for analyzing raw, real-time YouTube comments. This approach has proven to reduce neutral classifications by 96%, leading to more actionable insights from live data feeds.
This system is part of a broader Unified Scoring System and is primarily invoked during live analysis scenarios, such as the "Live Comments" page on the dashboard. It contrasts with the more conservative SVM model used for pre-processed, static datasets.
Sources: README.md, DEPLOYMENT_SUMMARY.md, UNIFIED_SCORING_SYSTEM.md
Core Logic and Architecture
The core of this feature is the AggressiveSentimentAnalyzer class, which employs a rule-based system with expanded patterns and a low tolerance for neutrality.
Sources: src/modeling/aggressive_analyzer.py
Analysis Flow
The analyze_sentiment method follows a specific flow to classify text. It calculates scores for positive and negative indicators, boosts them based on intensifiers, and then makes a decisive classification.
The diagram below illustrates the decision logic within the analyze_sentiment function.
Key Components
The analyzer is built on several key components that work together to achieve its aggressive classification behavior.
| Component | Description | Source File |
|---|---|---|
| Positive Indicators | A list of regex patterns to detect positive words, expressions, and emojis. | aggressive_analyzer.py:8-25 |
| Negative Indicators | An expanded list of regex patterns for negative words, context, criticism, and emojis. | aggressive_analyzer.py:28-50 |
| Question Patterns | Patterns to identify questions, which are often treated as indicators of dissatisfaction or negative sentiment. | aggressive_analyzer.py:53-56 |
| Intensifiers | Words that boost the weight of other indicators (e.g., "sangat", "banget"). | aggressive_analyzer.py:59-61 |
| Neutral Threshold | A very low threshold (0.1) that forces a classification unless scores are nearly identical. | aggressive_analyzer.py:86 |
Scoring Mechanism
- Indicator Counting: The system counts the occurrences of each pattern type in the input text.
- Intensifier Boost: A
intensifier_boostis calculated. The base is1.0, and it increases by0.3for each intensifier found. - Score Calculation:
positive_scoreis the count of positive indicators multiplied by theintensifier_boost.negative_scoreis the sum of negative indicators and half the count of question patterns, all multiplied by theintensifier_boost.
- Decision:
- If one score is greater than the other by more than the
neutral_threshold(0.1), that sentiment is chosen. - If the scores are too close, the system defaults to a "lean negative" logic, classifying the text as negative if it contains questions, uncertainty words like "tapi" (but), or is longer than 5 words. This is based on the assumption that ambiguous or longer comments in this context are more likely to be complaints.
- If one score is greater than the other by more than the
Sources: src/modeling/aggressive_analyzer.py:79-99
Integration with the Unified Scoring System
The Aggressive Sentiment Analysis is not a standalone feature but is integrated into the application's Unified Scoring System. This system defines different scoring behaviors for different contexts.
The diagram below shows the different modes within the Unified Scoring System.
The "Realtime Mode" directly corresponds to the aggressive analysis logic, designed to boost clear sentiment signals and reduce neutral outcomes for live comment feeds.
Sources: UNIFIED_SCORING_SYSTEM.md
API Integration and Data Flow
The aggressive analysis is triggered via API calls from the frontend dashboard, particularly for features that handle live data.
The sequence diagram below shows how a request for live comments from the Next.js dashboard results in the use of the aggressive ("realtime") analysis model.
The key step is the live-comments API endpoint calling the /predict endpoint with model_type: "realtime". This explicitly requests the aggressive scoring behavior for each comment.
Sources: dashboard-next/app/api/live-comments/route.ts:182-186, UNIFIED_SCORING_SYSTEM.md
Fallback Mechanism
In cases where the primary prediction API (/predict) is unavailable, the /api/live-comments endpoint contains its own analyzeSentimentFallback function. This function uses a similar, but distinct, weighted lexicon and thresholding logic to ensure that sentiment analysis can still be performed, providing system resilience.
Sources: dashboard-next/app/api/live-comments/route.ts:7-60, 201-207
Analysis Output
The analyze_sentiment method returns a dictionary containing the classification and metadata about the analysis.
| Key | Type | Description |
|---|---|---|
sentiment | str | The resulting sentiment: 'positive', 'negative', or 'neutral'. |
confidence | float | A score indicating the confidence of the classification, typically between 0.5 and 0.95. |
reasoning | str | A brief explanation for the classification (e.g., "Positive indicators: 2, boost: 1.3"). |
Sources: src/modeling/aggressive_analyzer.py:109-113
Conclusion
The Aggressive Sentiment Analysis feature is a critical component for deriving meaningful, real-time insights from user comments. By intentionally minimizing neutral results and making decisive classifications based on expanded contextual patterns, it transforms raw, noisy data into a clear sentiment signal. Its integration into the dual-model architecture and the Unified Scoring System allows it to be applied specifically where it's most effective—on live, unprocessed data—contributing to a 96% reduction in neutral results and enhancing the overall value of the analytics dashboard.
Sources: DEPLOYMENT_SUMMARY.md, README.md