How It Works (High-Level)

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• docker-compose.prod.yml
• src/app.ts
• src/commands/join.ts
• src/commands/start.ts
• src/env.ts
• src/interactions.ts
• src/processors/readText.ts
• src/processors/recognizeRecording.ts
• src/processors/recordAudio.ts
• src/processors/translate.ts
• src/utils/AudioQueue.ts
• stt/src/main.rs
• package.json
• requirements.txt

The Discord Live Translator bot provides real-time translation of voice chat within a Discord channel. It listens to a user speaking, transcribes their speech to text, translates that text into a target language selected by another user, and then reads the translated text aloud in the voice channel. This allows for seamless communication between users who speak different languages.

The system is built on a distributed architecture, comprising several independent services that work together to form a translation pipeline. The main components are the Node.js-based Discord bot, a Rust-based Speech-to-Text (STT) service, an instance of LibreTranslate for translation, and a Python script for Text-to-Speech (TTS) synthesis. These services are orchestrated using Docker Compose for ease of deployment.

Sources: README.md, docker-compose.prod.yml

System Architecture

The project is composed of four main services that are managed via Docker. The docker-compose.prod.yml file defines the relationship between these services. The main bot service communicates with the stt and libretranslate services to perform its functions, and uses redis for caching user settings.

Sources: docker-compose.prod.yml

The following diagram illustrates the services defined in the production Docker Compose configuration.

Service	Image	Role
bot	ghcr.io/wotschofsky/discord-live-translator	The main Discord bot application. Connects to Discord, manages voice, and orchestrates other services.
stt	ghcr.io/wotschofsky/discord-live-translator/stt	Speech-to-Text service. Transcribes received audio files into text using the Whisper model.
libretranslate	libretranslate/libretranslate:v1.3.11	Text translation service. Translates text from a source language to a target language.
redis	redis:alpine	In-memory data store used for caching user translation preferences.

Sources: docker-compose.prod.yml, src/commands/start.ts:33

Core Translation Flow

The translation process is a multi-step pipeline that begins when a user speaks in a voice channel where the bot is active. The flow involves audio capture, speech recognition, translation, and speech synthesis.

Sources: src/commands/join.ts:81-99

This sequence diagram shows the interaction between the user, Discord, and the various backend services during a single translation event.

Key Components & Processes

Bot Initialization & Command Handling

The application's entry point is src/app.ts. It initializes the Discord client with the necessary intents (Guilds, GuildVoiceStates) to see servers and listen to voice activity. An interactionCreate event listener is registered to route slash command interactions to their respective handlers.

Sources: src/app.ts:10-11, 16-41

On startup, if GUILD_ID is set, the bot registers its slash commands for that specific guild. The commands are defined in the src/commands directory and collected in src/interactions.ts.

Sources: src/app.ts:51-57, src/interactions.ts:13-26

Command	Description	Source File
/join	Move the bot into your voice channel	src/commands/join.ts:19-21
/leave	Move the bot out of your voice channel	src/commands/leave.ts:10-12
/start	Start the translation and select a language	src/commands/start.ts:13-15
/stop	Stop the translation	src/commands/stop.ts:6-8
/status	Show your current translation settings	src/commands/status.ts:8-10
/languages	Show all available languages	src/commands/languages.ts:6-8

Audio Recording & Processing

When a user starts speaking, the speaking event on the voice connection's receiver is triggered. This calls the recordAudio function.

1. A voice receiver subscribes to the user's audio stream. Recording stops after 100ms of silence (EndBehaviorType.AfterSilence).
2. The incoming Opus audio packets are decoded into raw PCM data using the opusscript library.
3. The raw audio data is written to a temporary .wav file with a sample rate of 16000 Hz and 1 channel (mono).

Sources: src/commands/join.ts:83, src/processors/recordAudio.ts:28-34, 39-43

Speech-to-Text (STT) Service

The STT service is a Rust-based web server built with actix-web. It exposes a single endpoint / that accepts POST requests with audio/wav content.

• Model: It uses the whisper-rs library, a Rust binding for OpenAI's Whisper model. The model file is loaded from the path specified in the MODEL_PATH environment variable.
• Processing: Upon receiving a .wav file, it reads the audio samples, converts them to 32-bit floats, and processes them through the Whisper model.
• Forced English Translation: The service is hardcoded to always translate the recognized speech into English. This simplifies the downstream translation process by providing a consistent source language.

// stt/src/main.rs:31-32
params.set_translate(true);
params.set_language(Some("en"));

Sources: stt/src/main.rs, src/processors/recognizeRecording.ts:9-15

Text Translation

If the user's target language is not English, the bot sends the English text from the STT service to the translation service (libretranslate). This is done via a POST request to the /translate endpoint, specifying the text (q), source language (en), and the target language code.

Sources: src/commands/join.ts:93-95, src/processors/translate.ts:10-19

Text-to-Speech (TTS) & Audio Playback

The final step is converting the translated text back into speech.

1. The readText function calls an external Python script (src/lib/tts/read_text.py) using execa.
2. It passes the TTS model name corresponding to the target language, the text to be spoken, and an output file path.
3. The Python script (which relies on libraries in requirements.txt like TTS) generates a new .wav file.
4. This file is then added to the AudioQueue, which manages playback in the voice channel. The queue ensures that audio clips are played one after another without interrupting each other.

Sources: src/processors/readText.ts:10-20, src/utils/AudioQueue.ts:48-52

Configuration

The system's behavior is controlled by environment variables, which are parsed in src/env.ts using @t3-oss/env-core and Zod for validation.

Variable	Description	Required	Default Value
BOT_TOKEN	The Discord bot token.	Yes	-
CLIENT_ID	The Discord application client ID.	No	-
GUILD_ID	If set, commands are deployed to this guild on startup.	No	-
STT_HOST	The URL of the Speech-to-Text service.	Yes	-
TRANSLATION_HOST	The URL of the LibreTranslate service.	Yes	-
REDIS_URL	The connection URL for the Redis instance.	No	-
HEARTBEAT_URL	A URL to send a POST request to every minute as a health check.	No	-
LOG_PATH	A file path to write logs to.	No	-
POSTHOG_API_KEY	API key for PostHog analytics.	No	-

Sources: src/env.ts:8-22, docker-compose.prod.yml:8-11

Summary

The Discord Live Translator operates as a sophisticated pipeline, leveraging a microservices architecture to handle the complex task of real-time voice translation. It effectively decouples audio processing (Node.js), speech recognition (Rust), translation (LibreTranslate), and speech synthesis (Python), allowing each component to be specialized for its task. The use of slash commands for user interaction and a central audio queue for playback provides a robust and user-friendly experience.

Sources: README.md, docker-compose.prod.yml, src/app.ts