Text-to-Speech (TTS)
Relevant source files
The following files were used as context for generating this wiki page:
Text-to-Speech (TTS) converts text into audible speech using on-device inference engines. Within the RunAnywhere SDK, TTS operates as a modality powered by the C++ core (runanywhere-commons), which dispatches synthesis requests to specialized backends like Sherpa-ONNX (Piper) or platform-specific system voices.
Sources: core/README.md, AGENTS.md
The system supports two primary modes of operation: batch synthesis, which returns a complete audio buffer, and streaming synthesis, which yields audio chunks incrementally as the engine produces them. Higher-level abstractions also provide direct device playback (the speak API), which manages the lifecycle of synthesis and audio output through a SpeechHandle.
Sources: bindings/electron/src/api/speech.ts:474-510, bindings/web/packages/core/src/Public/API/Namespaces/tts.ts:114-143
Architecture and Components
The TTS system follows a provider-based architecture where the C++ core acts as a router. Platform SDKs (Electron, Web, Kotlin, Swift) communicate with the core via a stable C ABI (rac_*).
Data Flow Overview
The following diagram illustrates the flow from a user's text input to the final audio output.
Key Engines and Backends
| Engine | Description | Primitive Slot |
|---|---|---|
| Sherpa-ONNX | Provides offline TTS using Piper/VITS models. | tts_ops |
| QHexRT | Accelerates TTS on Qualcomm Hexagon NPUs (Android/Windows-ARM). | tts_ops |
| Platform | Utilizes native system TTS engines (Apple AVSpeechSynthesizer). | tts_ops |
Sources: core/README.md, AGENTS.md
API Surface and Operations
The TTS functionality is exposed through the tts namespace in the SDKs.
Synthesis Methods
synthesize(text, options): Performs batch synthesis. It returns anAudioobject containing Float32 samples, sample rate, and duration.
Sources: bindings/electron/src/api/speech.ts:512-526, bindings/web/packages/core/src/Public/API/Namespaces/tts.ts:114-124synthesizeStream(text, options): Returns anAsyncIterableIteratorofAudioChunkobjects. This allows the application to process or play audio before the entire synthesis is complete.
Sources: bindings/electron/src/api/speech.ts:528-550, bindings/web/packages/core/src/Public/API/Namespaces/tts.ts:131-143speak(text, options): A high-level API that synthesizes and immediately plays the audio through the system speakers. It returns aSpeechHandle.
Sources: bindings/electron/src/api/speech.ts:578-582, bindings/web/packages/core/src/Public/API/Namespaces/tts.ts:153-157
Speech Management
The SpeechHandle allows developers to manage active playback.
Configuration and Options
TTS behavior is customized using the TtsOptions structure.
| Parameter | Type | Description |
|---|---|---|
voice | string | The ID of the voice to use for synthesis. |
speed | number | The speaking rate (e.g., 1.1 for 10% faster). |
languageCode | string | BCP-47 language tag for voice selection. |
pitch | number | Adjusts the tone of the synthesized voice. |
Sources: bindings/electron/src/api/speech.ts:600-610, bindings/electron/src/api/types.ts:316-320
Audio Output Format
The SDK typically produces raw PCM audio. In the Electron and Web SDKs, this is normalized to Float32Array samples in the range [-1, 1] for compatibility with the Web Audio API.
Sources: bindings/electron/src/api/speech.ts:517-525, bindings/web/packages/core/src/Public/API/Namespaces/tts.ts:36-50
Implementation Details
Playback Lifecycle (Web/Electron)
Direct playback via speak() uses an internal Playback class that manages a singleton AudioContext. This ensures that only one utterance plays at a time and allows for immediate interruption.
Error Handling
TTS operations throw SDKException when failures occur, such as when no voice model is loaded. For the speak() API, errors are captured within the SpeechHandle.error property to prevent unhandled promise rejections during asynchronous playback.
Sources: bindings/electron/src/api/speech.ts:568-571, bindings/web/packages/core/src/Public/API/Namespaces/tts.ts:98-101
Summary
The TTS system provides a flexible, cross-platform interface for generating high-quality speech on-device. By abstracting complex engine logic behind a simple API and providing both batch and streaming modes, the SDK enables responsive voice interactions in applications while maintaining privacy and offline capability.