Cloud LLM Providers Integration
Relevant source files
The following files were used as context for generating this wiki page:
The Cloud LLM Providers Integration in mausVoice facilitates the enhancement of raw transcriptions using remote Large Language Models. While the application supports local execution via Whisper, it integrates with external providers like Groq, OpenAI, and OpenRouter to perform "AI cleanup"—a process that removes filler words, applies punctuation, and formats text according to specific writing styles or "tones".
This system acts as the "Brain" of the application, where business logic resides in TypeScript and interacts with external APIs to transform raw audio data into polished text. The integration ensures that even complex speech patterns, self-corrections, and technical jargon are accurately processed before being typed into the user's focused application.
Sources: llms.txt, README.md, apps/desktop/test/evals/post-processing-eval.test.ts:36-60
Architecture and Data Flow
The integration follows a "Rust is the API, TypeScript is the Brain" pattern. When a user finishes speaking, the raw transcription is passed to the AI cleanup module. The system constructs a prompt based on the user's chosen writing style and sends it to the configured cloud provider.
The diagram above illustrates the selection process between different cloud providers to process a raw transcription into cleaned text.
Post-Processing Logic
The core of the integration is the postProcess function, which orchestrates the interaction with the LLM. It utilizes utility functions to build both system and user prompts, ensuring the LLM understands the specific constraints of the requested writing style.
Key components involved in this flow include:
- Prompt Builders:
buildSystemPostProcessingTonePromptandbuildPostProcessingPromptcreate the instructions for the LLM. - Schema Validation: The system uses
PROCESSED_TRANSCRIPTION_JSON_SCHEMAto ensure the cloud provider returns a valid JSON object. - Text Generation: The
generateTextmethod of the selected provider's repository handles the actual API call.
Sources: apps/desktop/test/evals/post-processing-eval.test.ts:36-60
Provider Capabilities
The project integrates multiple cloud providers, each offering distinct advantages for text cleanup and formatting.
| Provider | Primary Use Case | Key Features |
|---|---|---|
| Groq | High-speed AI cleanup | Support for low-latency Llama and Mixtral models; encrypted keys. |
| OpenRouter | Model flexibility | Unified API to access various LLM models for text processing. |
| OpenAI | Standard cleanup/formatting | Reliable processing for transcription cleaning and style application. |
| Deepgram | Streaming transcription | Real-time nova-3 transcription (separate from LLM cleanup). |
Sources: llms.txt, README.md, packages/voice-ai/package.json
Writing Styles and Tones
Cloud LLMs are specifically leveraged to apply different "tones" to the transcribed text. This is managed through a ToneConfig which guides the LLM on how to rewrite the transcript.
| Style | LLM Objective |
|---|---|
| Default | Remove filler words (ums/likes), fix grammar, improve readability. |
| Format with greetings, sign-offs, and professional structure. | |
| Chat | Format as casual text messages; remove formal structures. |
| Formal | Remove contractions and slang; suitable for official documents. |
| Custom | User-defined prompts (e.g., Customer Support, Motivational Coach). |
Sources: apps/desktop/test/evals/post-processing-eval.test.ts:85-300
Implementation Details
API Interaction Sequence
The following sequence diagram detail how the TypeScript layer interacts with a Cloud LLM provider (using Groq as the primary example) to clean up a transcription.
This diagram shows the synchronous flow of data from prompt construction to schema validation when interacting with an LLM.
Sources: apps/desktop/test/evals/post-processing-eval.test.ts:47-60
Security and Encryption
API keys for providers like Groq and OpenAI are never hardcoded or baked into the binary. Instead, they are entered by the user in the application settings and stored on disk using XChaCha20-Poly1305 encryption.
Evaluation and Quality Control
The integration is verified through comprehensive evaluation tests (post-processing-eval.test.ts). These tests simulate various scenarios to ensure the Cloud LLMs accurately handle:
- Self-Corrections: Replacing "Monday... no wait Wednesday" with just "Wednesday".
- Code Terms: Properly formatting identifiers like
eval.utils.tswith backticks. - Special Characters: Converting spoken "newline" or "open paren" into structural symbols.
- Multilingual Support: Cleaning transcripts in German, Spanish, Chinese, and Portuguese.
Sources: apps/desktop/test/evals/post-processing-eval.test.ts:160-260
The integration of Cloud LLM Providers transforms mausVoice from a simple dictation tool into an intelligent text processor. By leveraging the low-latency processing of providers like Groq, the system delivers polished, context-aware text while maintaining user privacy through local key encryption and optional offline fallbacks.