Voice & Emotional Analysis
Relevant source files
The following files were used as context for generating this wiki page:
The Voice & Emotional Analysis system within the TAMV MD-X4™ ecosystem is a core component of the Isabella AI assistant. It provides a multi-layered approach to human-computer interaction by combining real-time dimensional emotional analysis with sophisticated text-to-speech (TTS) and immersive audio synthesis. This system is designed to create a bridge between users and the platform's capabilities through empathetic, scientifically-grounded feedback and high-quality vocal responses.
The scope of this system includes the processing of user input to detect psychological states, the orchestration of LLM-generated responses via a phrase-level streaming pipeline, and the delivery of those responses through a low-latency audio engine. It integrates closely with other domains such as Dream Spaces and the KAOS Audio System to provide a unified sensory experience.
Sources: supabase/functions/isabella-chat-enhanced/index.ts:31-104, PLAN-TAMV-MODULAR.md:73-81
Emotional Analysis Architecture
The emotional detection logic is built upon Russell's Circumplex Model (1980), which maps affective states across three primary dimensions: Valence, Arousal, and Dominance. This allows the system to categorize user input into specific quadrants, such as "excited_alert" or "depressed_lethargic," to tailor Isabella's interventions.
Dimensional Model Details
The system calculates these dimensions by scanning user text against validated dictionaries of emotional keywords based on the research of Ekman, Plutchik, and Russell.
| Dimension | Range | Description |
|---|---|---|
| Valence | -1.0 to +1.0 | Indicates whether the state is negative or positive. |
| Arousal | -1.0 to +1.0 | Indicates the level of activation (calm vs. excited). |
| Dominance | -1.0 to +1.0 | Indicates the sense of control vs. submission. |
| Confidence | 0.0 to 1.0 | Normalized score based on the number of keyword matches detected. |
Sources: src/hooks/useIsabellaEmotionalAnalysis.ts:4-56, src/hooks/useIsabellaEmotionalAnalysis.ts:98-132
Emotional Quadrants and Interventions
Based on the calculated Valence and Arousal, the system assigns a quadrant and recommends a specific clinical intervention strategy, such as grounding techniques for anxiety (Linehan, 1993) or behavioral activation for sadness (Beck, 1979).
The diagram shows the flow from raw user text to the categorization of emotional states and the generation of a tailored response strategy.
Sources: src/hooks/useIsabellaEmotionalAnalysis.ts:134-192
Vocal Synthesis and Audio Pipeline
The "Isabella Prime" specification defines a robust pipeline for converting text responses into speech using a phrase-level synchronization protocol. This ensures that audio playback begins as soon as the first coherent "chunk" of text is generated by the LLM.
Chunk-by-Phrase Streaming
Instead of waiting for an entire response, the system breaks text into chunks based on punctuation marks (., !, ?, or long commas). Each chunk is then processed independently through the TTS engine.
Sources: 02_MODULOS/M05_IA_TAMVAI/INTERNO/ISABELLA-PRIME-SPEC.md:25-36
TTS Cache System
To minimize latency and API costs, the system utilizes a PostgreSQL-based cache (tts_cache). It stores audio URLs indexed by a SHA256 hash of the combined text and voice ID.
This sequence diagram illustrates the low-latency fallback logic used to optimize vocal responses.
Sources: 02_MODULOS/M05_IA_TAMVAI/INTERNO/ISABELLA-PRIME-SPEC.md:46-77
Immersive Audio Engine
The ImmersiveAudioEngine is a singleton class responsible for managing background tracks, spatial effects, and browser-native speech synthesis (Web Speech API) as a secondary layer to the high-fidelity Isabella TTS.
Voice Profiles
The system supports multiple vocal "personalities" for Isabella, configured with varying rates, pitches, and acoustic effects like echo and reverb.
| Profile | Rate | Pitch | Echo | Reverb |
|---|---|---|---|---|
| isabella-human | 0.92 | 0.95 | 0.3 | 0.4 |
| isabella-epic | 0.85 | 0.88 | 0.5 | 0.6 |
| isabella-calm | 0.95 | 1.00 | 0.2 | 0.3 |
| isabella-leader | 0.88 | 0.90 | 0.4 | 0.5 |
Sources: src/systems/AudioSystem.ts:74-106
Audio Node Structure
The engine uses the Web Audio API to create a sophisticated signal chain for vocal and ambient sounds.
The diagram represents the audio processing chain where raw sounds are enhanced with spatial and temporal effects before output.
Sources: src/systems/AudioSystem.ts:133-172
Operational Performance Targets
The system is held to strict performance and reliability standards defined in the modular plan and Isabella Prime specifications.
- P95 Latency: The system aims for a response time (chat + audio) of under 4-5 seconds.
- Cache Hit Rate: Target production cache efficiency is >60%.
- Fallback Logic: If the high-fidelity TTS (ElevenLabs) fails within 8 seconds, the system falls back to text-only display or local Web Speech API synthesis to prevent UI crashes.
- TTS Fallback Rate: Target failure rate is <5%.
Sources: 02_MODULOS/M05_IA_TAMVAI/INTERNO/ISABELLA-PRIME-SPEC.md:92-111, PLAN-TAMV-MODULAR.md:83-91
Conclusion
The Voice & Emotional Analysis module is a sophisticated integration of affective computing and audio engineering. By utilizing a dimensional model for psychological analysis and a chunk-based caching system for vocal synthesis, TAMV MD-X4™ ensures that its primary AI assistant, Isabella, remains both technologically efficient and emotionally resonant with its users.