AI Text & Stylometry Detection
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
AI Text & Stylometry Detection is a core component of the TrueLens platform designed to identify AI-generated content and verify linguistic authenticity. The system utilizes a multi-signal approach, combining deep-learning transformer models with traditional stylometric heuristics to differentiate between human-written and machine-generated text.
This system serves as the primary "Text" signal in the overall ensemble trust score, providing critical evidence for content veracity in news, product reviews, and general web documents. Sources: truelens-backend/app/services/text_analysis.py:1-12, truelens-app/src/lib/analysis.ts:5-8
Architecture and Ensemble Logic
The detection engine operates as an ensemble of two primary methodologies: Transformer-based Analysis and Advanced Stylometry. By default, the system weights these signals to produce a final "AI Probability" score.
- Transformer Signal (70%): Utilizes fine-tuned RoBERTa models to detect neural network patterns.
- Stylometry Signal (30%): Evaluates linguistic "burstiness," vocabulary diversity, and structural uniformity.
If the transformer infrastructure is unavailable, the system gracefully falls back to a 100% stylometry-based analysis. Sources: truelens-backend/app/services/text_analysis.py:270-285, truelens-backend/app/tasks/ml_pipeline.py:133-140
The following diagram illustrates the ensemble data flow during a text scan:
This diagram shows how different detection signals are weighted and combined to form the final result. Sources: truelens-backend/app/services/text_analysis.py:270-300, truelens-backend/app/tasks/ml_pipeline.py:43-55
Stylometric Feature Extraction
Stylometry focuses on the unique "fingerprint" of human writing. The system extracts specific metrics where human text typically shows higher variance compared to the uniform output of Large Language Models (LLMs).
Key Stylometric Metrics
| Feature | Description | AI Signature |
|---|---|---|
| Burstiness | Variation in paragraph lengths and structure. | Low (Uniformly sized paragraphs) |
| Sentence Variance | Consistency of sentence lengths within a text block. | Low (Uniform sentence lengths) |
| Vocabulary Richness | The ratio of unique words to total word count (Type-Token Ratio). | Medium-Low (Repetitive/Uniform vocab) |
| Hapax Legomena | Ratio of words that appear exactly once in the text. | Low (Fewer unique "one-off" words) |
| Connective Ratio | Overuse of formal connectives (e.g., "Furthermore", "Moreover"). | High (Overly formal structure) |
| Punctuation Density | Distribution of commas, semicolons, and periods per word. | Very High or Very Low |
Sources: truelens-backend/app/services/text_analysis.py:75-155, truelens-app/src/lib/analysis.ts:18-75
Implementation Example (Python)
The backend utilizes re and math libraries to calculate these linguistic signatures:
# Calculate Burstiness (variation in paragraph lengths)
paragraphs = [p.strip() for p in text.split('\n\n') if p.strip()]
para_lengths = [len(p.split()) for p in paragraphs] if paragraphs else [num_words]
avg_para_len = sum(para_lengths) / max(len(para_lengths), 1)
burstiness = (
math.sqrt(sum((x - avg_para_len) ** 2 for x in para_lengths) / max(len(para_lengths), 1))
/ max(avg_para_len, 1) if len(para_lengths) > 1 else 0.5
)
Sources: truelens-backend/app/services/text_analysis.py:93-104
Transformer-Based Detection
The transformer pipeline leverages state-of-the-art models fine-tuned specifically for AI detection. The system attempts to load models in the following priority:
roberta-base-openai-detector(OpenAI's GPT-2 detector)Hello-SimpleAI/chatgpt-detector-roberta
The pipeline truncates text to 512 tokens to match the model's context window and normalizes the output into a probability score (0-100), where 100 indicates a "Fake" (AI-generated) label. Sources: truelens-backend/app/services/text_analysis.py:34-58, truelens-backend/app/services/text_analysis.py:228-255
Specialized Content Analyzers
TrueLens extends text detection with domain-specific analyzers that look for behavioral indicators of inauthenticity.
News Integrity Analysis
The news_analysis service identifies "Fake News" by looking for sensationalism and clickbait framing. It checks for:
- Sensationalism Index: Keywords like "shocking," "mind-blowing," or "conspiracy."
- Citation Check: Presence of attribution phrases like "according to" or "researchers at."
- Emotional Manipulation: Excessive exclamation marks and unprofessional capitalization.
Sources: truelens-backend/app/services/news_analysis.py:46-95
Review Authenticity Analysis
The review_analysis service identifies fraudulent product/business reviews using:
- Sentiment Extremity: Identifying 1 or 5-star "polar" reviews that lack nuanced language.
- Cliche Matching: Searching for common paid-review phrases like "best on the market" or "highly recommend."
- Lexical Diversity: Low diversity often signals keyword stuffing for SEO purposes.
Sources: truelens-backend/app/services/review_analysis.py:45-90
Data Flow and Processing
Text analysis is orchestrated through a Celery task pipeline to ensure asynchronous processing.
This diagram outlines the lifecycle of a text scan from the initial user submission to final result storage. Sources: truelens-backend/app/tasks/ml_pipeline.py:100-210, truelens-app/src/app/scan/page.tsx:38-65, truelens-app/src/app/developers/page.tsx:18-35
Summary
AI Text & Stylometry Detection provides a robust, multi-layered defense against synthetic content. By combining the pattern-matching power of Transformers with the structural analysis of Stylometry, TrueLens achieves high accuracy across diverse text formats, including news, reviews, and general web content. The system's weighting logic ensures that text analysis remains the most significant contributor (40%) to the platform's aggregate trust score. Sources: truelens-backend/app/tasks/ml_pipeline.py:32-40, truelens-backend/app/services/text_analysis.py:1-10