Hybrid and Mimic Models

Relevant source files

The following files were used as context for generating this wiki page:

• src/modeling/gemini_model_mimic.py
• src/modeling/gemini_svm_mimic.py
• src/modeling/static_model_mimic.py
• src/api/gemini_svm_api.py
• STATIC_MODEL_MIMIC.md
• DUAL_MODEL_ARCHITECTURE.md
• UNIFIED_SCORING_SYSTEM.md

The sentiment analysis system employs a dual-model architecture, utilizing a traditional SVM model for static, historical data and Google's Gemini AI for live, real-time analysis. This approach creates a potential for inconsistent results between the static and live dashboards. To solve this, the project introduces "Mimic Models"—a sophisticated system where Gemini AI is specifically prompted and trained to replicate the behavior, patterns, and decision logic of the static models, ensuring analytical consistency across the entire platform.

This hybrid approach leverages the speed and consistency of SVM for batch processing while harnessing the contextual understanding of Gemini AI for live analysis, with the mimic layer acting as a bridge to unify the user experience. For more information on the base architecture, see the Dual Model Architecture documentation.

Sources: DUAL_MODEL_ARCHITECTURE.md, STATIC_MODEL_MIMIC.md

System Architecture

The Mimic System is designed to make the live dashboard's AI-powered analysis behave like the static dashboard's SVM-based analysis. It analyzes the patterns and thresholds of the static models and applies them as constraints or examples for the Gemini AI, resulting in consistent sentiment classification.

The following diagram illustrates the role of the Mimic System in unifying the static and live analysis paths.

Mimic Model Implementations

The project contains several Python classes dedicated to mimicking the static models. Each uses a different strategy to achieve behavioral consistency.

StaticModelMimic

This class implements an enhanced rule-based system that mimics the static models' conservative behavior without making an LLM call for every prediction. It relies on pre-defined sentiment patterns, weights, and conservative thresholds derived from analyzing the static SVM model.

• Logic: It calculates scores based on strong, medium, and weak sentiment patterns. A conservative_factor of 0.85 is applied to reduce extreme scores, and classification is determined by conservative thresholds (e.g., pos_ratio > 0.4), which favors neutral outcomes, similar to the original SVM model.
• Use Case: A lightweight, fast, and dependency-free way to get consistent analysis for the live dashboard, especially when Gemini API calls are not desirable.

The data flow for this model is as follows:

GeminiSVMMimic

This class uses a few-shot learning approach to train Gemini AI to specifically replicate the output of the primary svm_model.pkl.

• Logic:
  1. generate_training_examples: It first uses the loaded SVM model to generate a set of high-confidence predictions on sample texts.
  2. create_few_shot_prompt: These examples are formatted into a detailed prompt, instructing Gemini to analyze a new target text "in the same style as the SVM model."
  3. predict_like_svm: Gemini AI processes the prompt and returns a prediction that mimics the SVM's likely output, including a confidence score and reasoning.
• Use Case: High-fidelity replication of the SVM model's behavior, leveraging Gemini's pattern-matching capabilities.

The class structure and its main methods are shown below.

GeminiModelMimic

This is a more generalized version that can mimic an ensemble of static models, not just the SVM pipeline. It loads the svm_pipeline, fixed_models, tfidf_vectorizer, and label_encoder.

• Logic: It generates predictions from all loaded static models to create a rich set of training examples. The prompt (create_mimic_prompt) is more complex, showing Gemini the outputs from both the "SVM Pipeline" and the "Fixed Models" ensemble, asking it to produce a final prediction that considers both.
• Use Case: Ensuring consistency with the entire suite of static models used in the dashboard.

The prediction flow involves generating examples from multiple models before prompting Gemini.

API Integration

The GeminiSVMMimic functionality is exposed through a FastAPI application, allowing other services to request SVM-style predictions without needing to manage the model loading and prompting logic.

Sources: src/api/gemini_svm_api.py

API Endpoints

The following table summarizes the key endpoints provided by gemini_svm_api.py.

Endpoint	Method	Description
/health	GET	Health check for the API service.
/predict-svm-style	POST	Predicts sentiment for a single text using the Gemini SVM mimic model.
/compare-with-svm	POST	Compares Gemini mimic predictions against the original SVM for a batch of texts.
/batch-train-predict	POST	Uses the first half of a batch to generate training examples and predicts on the second half.
/test-mimic	GET	Runs a pre-defined test to check the agreement rate between the mimic and original SVM model.

Sources: src/api/gemini_svm_api.py:29-100

Prediction Flow

This sequence diagram shows the interaction when a client calls the /predict-svm-style endpoint.

Fallback Mechanisms

To ensure robustness, especially if the Gemini API is unavailable or fails, the mimic models include a fallback mechanism. If a prediction from Gemini cannot be obtained, the system defaults to making a direct prediction using the locally loaded static SVM model. This ensures that the service can still provide a result, albeit without the mimicry layer.

The _fallback_prediction method in GeminiModelMimic demonstrates this:

# src/modeling/gemini_model_mimic.py:151-174
def _fallback_prediction(self, text: str) -> Dict:
    """Fallback to direct static model prediction."""
    if 'svm_pipeline' in self.models:
        try:
            svm_model = self.models['svm_pipeline']
            prediction = svm_model.predict([text])[0]
            probabilities = svm_model.predict_proba([text])[0]
            confidence = max(probabilities)
            
            return {
                # ... (returns a result dictionary)
                'final_prediction': prediction,
                'mimic_confidence': confidence,
                'model': 'SVM-Direct'
            }
        except Exception as e:
            self.logger.error(f"SVM fallback error: {e}")
    
    return {
        # ... (returns a default neutral result)
        'model': 'Fallback'
    }

Sources: src/modeling/gemini_model_mimic.py:151-174, src/modeling/gemini_svm_mimic.py:126-146

Conclusion

The Hybrid and Mimic Models represent a key architectural feature of this project, designed to bridge the gap between a fast, deterministic SVM model and a powerful, contextual Gemini AI. By using various techniques—from rule-based logic to few-shot learning—the system ensures a consistent and predictable sentiment analysis experience across both historical and real-time data. The inclusion of robust fallback mechanisms and dedicated comparison utilities makes the system both reliable and maintainable.