Text-to-Speech (TTS) System (Python)
Relevant source files
The following files were used as context for generating this wiki page:The Text-to-Speech (TTS) system is a Python-based component responsible for converting translated text strings into audible speech. It is designed to be invoked as a separate process from the main Node.js application. This system utilizes the Coqui TTS library to synthesize voice and generate .wav audio files. These files are then queued and played back into a Discord voice channel, allowing users to hear translations in their selected language.
The system is composed of a primary script for on-demand text synthesis and a secondary script for pre-downloading the necessary language models. This decoupled architecture improves reliability, especially in locating the downloaded TTS models, and isolates the Python environment from the main Node.js application logic.
Sources: src/lib/tts/read_text.py:3, src/processors/readText.ts:8-10
Architecture and Data Flow
The TTS process is initiated by the TypeScript backend. When a text message needs to be spoken, the readText.ts processor executes the read_text.py Python script as a child process. It passes the text, the target language's model name, and an output file path as command-line arguments. The Python script then generates the audio file, which the TypeScript process subsequently adds to an audio playback queue for the corresponding voice connection.
The following diagram illustrates the high-level data flow for a single TTS request.
This diagram shows the handoff from the TypeScript environment to the Python environment for speech synthesis and the subsequent queuing of the generated audio file back in the TypeScript environment.
Sources: src/processors/readText.ts:18, src/lib/tts/read_text.py:17, src/utils/AudioQueue.ts:41
Sequence of Operations
The sequence diagram below details the interactions between the main application logic, the TTS processor, the Python script, and the file system.
This diagram shows that the readText.ts processor orchestrates the call to the Python script and then passes the resulting file path to the AudioQueue for playback.
Sources: src/commands/join.ts:98, src/processors/readText.ts:8-22, src/lib/tts/read_text.py:8-17
Core Components
read_text.py Script
This is the core script for generating audio from text. It is a command-line tool that accepts arguments to specify the model, text, and output file.
Command-Line Arguments
| Argument | Type | Required | Description |
|---|---|---|---|
--model | string | Yes | The name of the TTS model to use for synthesis. |
--text | string | Yes | The text to be converted to speech. |
--output | string | Yes | The file path where the generated .wav file will be saved. |
Sources: src/lib/tts/read_text.py:6-9
Implementation Details
The script uses the argparse library to handle command-line inputs. It initializes the TTS class from the Coqui library, loads the specified model by name (with GPU usage explicitly disabled), and calls tts_to_file to perform the synthesis and save the output.
# src/lib/tts/read_text.py:12-17
tts = TTS()
tts.load_tts_model_by_name(args.model, gpu=False)
tts.tts_to_file(args.text, file_path=args.output)
Sources: src/lib/tts/read_text.py:12-17
readText.ts Processor
This TypeScript module serves as the bridge between the Node.js application and the Python TTS script.
Function: readText
The readText function is an asynchronous function that takes a VoiceConnection, a message string, and a language key. Its primary responsibilities are:
- Constructing the path to the
read_text.pyscript. - Retrieving the correct
ttsModelname from thelanguages.tsconfiguration. - Escaping double quotes from the input message.
- Generating a random filename for the output
.wavfile in thecache/tts/directory. - Executing the Python script using
execawith the appropriate arguments. - Passing the generated audio file's path to the
audioQueuefor playback.
Sources: src/processors/readText.ts:8-22
Language and Model Configuration
The specific TTS models used for each language are defined in src/languages.ts and pre-downloaded by src/lib/tts/preload_models.py.
languages.ts
This file maps a language identifier (e.g., en, de) to its configuration, including the ttsModel string required by the Python script.
| Language | Display Name | TTS Model Name |
|---|---|---|
en | English | tts_models/en/ljspeech/vits |
de | German | tts_models/de/thorsten/vits |
fr | French | tts_models/fr/mai/tacotron2-DDC |
es | Spanish | tts_models/es/mai/tacotron2-DDC |
it | Italian | tts_models/it/mai_female/vits |
Sources: src/languages.ts:1-24
preload_models.py
This utility script is used to download all required TTS models from the internet. It iterates through a hardcoded list of model names and uses the tts.download_model_by_name method to fetch and cache them locally. This ensures that the models are available at runtime without incurring a download delay on the first request for a given language.
Sources: src/lib/tts/preload_models.py:3-13
Dependencies
The Python environment relies on several packages defined in requirements.txt. The most critical dependency for the TTS system is TTS==0.22.0.
Key dependencies include:
- TTS: The core Text-to-Speech library by Coqui.
- torch: A required backend for the TTS models.
- numpy: Used for numerical operations.
Sources: requirements.txt:1-1312
Summary
The Python-based TTS system is an essential but decoupled part of the discord-live-translator. It provides a robust mechanism for converting text to speech by leveraging the powerful Coqui TTS library. By running as an external process invoked from the main Node.js application, it isolates dependencies and simplifies the overall architecture. The system is configured through a central languages.ts file and relies on pre-downloaded models to ensure efficient, real-time audio generation.