/wotschofsky/discord-live-translator

Wiki directory

Browse AI-generated documentation for any repository.

Overall System Architecture

Relevant source files

The following files were used as context for generating this wiki page:

The Discord Live Translator is a bot designed to provide real-time translation of voice chat within a Discord channel. Its architecture is based on a set of containerized microservices that work together to capture, transcribe, translate, and synthesize speech. The central component is a Node.js application that acts as the Discord bot client, orchestrating the flow of data between the user and the various backend services.

This system is designed for self-hosting using Docker, with separate services for the main bot logic, speech-to-text (STT), translation, and data persistence. This modular approach allows each component to be managed and scaled independently. The core functionality is initiated by users through Discord slash commands, which control the bot's presence in voice channels and user-specific translation settings.

System Components

The application is composed of several distinct services and components, defined in the docker-compose.prod.yml file. These services communicate over a container network.

This diagram illustrates the services defined in the Docker Compose configuration and their dependencies.

Generating architecture diagram...

Sources: docker-compose.prod.yml

Service	Image	Role
`bot`	`ghcr.io/wotschofsky/discord-live-translator`	The main Node.js application. Connects to Discord, manages voice, and orchestrates the other services.
`stt`	`ghcr.io/wotschofsky/discord-live-translator/stt`	A Rust-based Speech-to-Text service using the Whisper model to transcribe audio.
`libretranslate`	`libretranslate/libretranslate:v1.3.11`	A self-hosted machine translation service used to translate the transcribed text.
`redis`	`redis:alpine`	A Redis instance for data persistence, likely used for storing user settings.

Sources: docker-compose.prod.yml:4-28

Main Bot Application (`bot`)

The bot service is the core of the system. It is a TypeScript/Node.js application responsible for all interactions with the Discord API.

Entry Point: The application starts with src/app.ts, which initializes the Discord client, sets up event listeners for interactions (slash commands), and logs into Discord.
Discord Client: Uses discord.js to connect to the Discord Gateway with Guilds and GuildVoiceStates intents.
Command Handling: A central interactionCreate event listener in src/app.ts routes incoming slash commands to their respective handlers (e.g., joinCommandHandler, startCommandHandler).
Voice Management: Uses @discordjs/voice to join, leave, and manage voice channel connections. It also receives user audio streams.
Orchestration: It coordinates the entire translation workflow by calling the stt, libretranslate, and local TTS components in sequence.

Sources: src/app.ts:3-19, src/commands/join.ts:59-71

Speech-to-Text Service (`stt`)

The stt service is a lightweight, high-performance web server written in Rust.

Framework: Built with actix-web.
Core Logic: Utilizes the whisper-rs library to run the Whisper speech recognition model. The model is loaded into memory on startup from the path specified by the MODEL_PATH environment variable.
API: It exposes a single endpoint:
- POST /: Accepts audio/wav data in the request body.
- It processes the audio, performs transcription (with translation to English enabled), and returns the resulting text as a plain string.
Performance: It is configured to use a maximum of 4 threads for processing to balance performance and resource usage.

Sources: stt/src/main.rs, stt/Cargo.toml:8-14

Translation Service (`libretranslate`)

The system uses a standard libretranslate Docker image for machine translation.

Functionality: The bot sends text to this service to be translated from English into a user's selected target language.
Configuration: The service is configured via the LT_LOAD_ONLY environment variable to only load specific language models (de, en, es, fr, it), saving memory.
API Interaction: The bot service communicates with it by sending a POST request to the /translate endpoint.

Sources: docker-compose.prod.yml:21-26, src/processors/translate.ts:12-21

Text-to-Speech (TTS) Component

Unlike other parts of the pipeline, TTS is not a standalone service. Instead, it is a Python script executed on-demand by the bot service.

Execution: The readText.ts module uses execa to run a Python script (src/lib/tts/read_text.py).
Functionality: The script takes a text string, a TTS model name, and an output file path as arguments. It uses the Coqui TTS (TTS) library to synthesize speech and save it as a .wav file.
Models: The specific TTS model used depends on the target language selected by the user, as defined in src/languages.ts.

Sources: src/processors/readText.ts:12-20, src/languages.ts:4-25

End-to-End Translation Flow

The entire translation process is initiated when a user speaks in a voice channel where the bot is active. The flow involves capturing audio, processing it through multiple services, and playing the final translated audio back into the channel.

The following sequence diagram illustrates the interactions between the components.

Generating architecture diagram...

Sources: src/commands/join.ts:110-126, src/processors/recordAudio.ts, src/processors/recognizeRecording.ts, src/processors/translate.ts, src/processors/readText.ts

Audio Capture: When a user with translation enabled speaks, the speaking event listener on the VoiceReceiver is triggered in joinCommandHandler.
Recording: The recordAudio processor is called. It subscribes to the user's audio stream, decodes the Opus packets into raw PCM data using OpusScript, and writes the data to a temporary .wav file. The recording automatically stops after a brief period of silence.
Speech-to-Text: The recognizeRecording processor reads the temporary .wav file and sends its contents in a POST request to the stt service. The service transcribes the audio to English text and returns it.
Translation: The joinCommandHandler checks the user's target language. If it is not English, the translate processor sends the transcribed text to the libretranslate service, which returns the translated text.
Text-to-Speech: The readText processor is called with the final text. It executes the read_text.py script, which generates a new .wav file containing the spoken version of the text.
Playback: The path to the newly generated audio file is added to the audioQueue, which plays it back into the voice channel for all members to hear.

Sources: src/commands/join.ts:110-126, src/processors/recordAudio.ts:29-41, src/processors/recognizeRecording.ts:8-17, src/processors/translate.ts:8-26, src/processors/readText.ts:12-20

Configuration

The system is configured using environment variables, which are parsed and validated in src/env.ts using @t3-oss/env-core and zod.

Variable	Type	Optional	Default	Description
`BOT_TOKEN`	`string`	No		The Discord bot token.
`CLIENT_ID`	`string`	Yes		The Discord application client ID.
`GUILD_ID`	`string`	Yes		If set, slash commands are deployed to this guild on startup.
`LOG_PATH`	`string`	Yes		If set, logs are appended to this file path.
`POSTHOG_API_KEY`	`string`	Yes		API key for PostHog telemetry.
`POSTHOG_HOST`	`string`	Yes	`https://app.posthog.com`	Host for PostHog telemetry.
`REDIS_URL`	`string`	Yes		URL for the Redis instance.
`STT_HOST`	`string`	No		Host URL for the Speech-to-Text service.
`TRANSLATION_HOST`	`string`	No		Host URL for the LibreTranslate service.
`HEARTBEAT_URL`	`string` (URL)	No		A URL to send a POST request to every minute as a heartbeat.

Sources: src/env.ts:9-21, docker-compose.prod.yml:9-12

Slash Commands

User interaction with the bot is handled exclusively through Discord slash commands.

Command Registration

Commands are defined using @discordjs/builders and collected in an array in src/interactions.ts. The initGuildCommands and initGlobalCommands functions use the @discordjs/rest module to register these commands with Discord's API.

If GUILD_ID is provided, commands are registered for that specific guild on bot startup.
A separate pnpm run deploy script (which runs src/deploy.ts) can be used to register commands globally.

Sources: src/interactions.ts, src/app.ts:51-57, src/deploy.ts

Available Commands

Command	Description
`/join`	Makes the bot join the user's current voice channel.
`/leave`	Makes the bot leave the voice channel.
`/start <target>`	Starts translation for the user, setting their target language.
`/stop`	Deactivates translation for the user.
`/status`	Shows the user's current translation status and target language.
`/languages`	Shows a list of all available target languages.

Sources: README.md:14-22, src/interactions.ts:12-18

Overall System Architecture

Relevant source files

The following files were used as context for generating this wiki page:

System Components

The application is composed of several distinct services and components, defined in the docker-compose.prod.yml file. These services communicate over a container network.

This diagram illustrates the services defined in the Docker Compose configuration and their dependencies.

Generating architecture diagram...

Sources: docker-compose.prod.yml

Service	Image	Role
`bot`	`ghcr.io/wotschofsky/discord-live-translator`	The main Node.js application. Connects to Discord, manages voice, and orchestrates the other services.
`stt`	`ghcr.io/wotschofsky/discord-live-translator/stt`	A Rust-based Speech-to-Text service using the Whisper model to transcribe audio.
`libretranslate`	`libretranslate/libretranslate:v1.3.11`	A self-hosted machine translation service used to translate the transcribed text.
`redis`	`redis:alpine`	A Redis instance for data persistence, likely used for storing user settings.

Sources: docker-compose.prod.yml:4-28

Main Bot Application (`bot`)

The bot service is the core of the system. It is a TypeScript/Node.js application responsible for all interactions with the Discord API.

Entry Point: The application starts with src/app.ts, which initializes the Discord client, sets up event listeners for interactions (slash commands), and logs into Discord.
Discord Client: Uses discord.js to connect to the Discord Gateway with Guilds and GuildVoiceStates intents.
Command Handling: A central interactionCreate event listener in src/app.ts routes incoming slash commands to their respective handlers (e.g., joinCommandHandler, startCommandHandler).
Voice Management: Uses @discordjs/voice to join, leave, and manage voice channel connections. It also receives user audio streams.
Orchestration: It coordinates the entire translation workflow by calling the stt, libretranslate, and local TTS components in sequence.

Sources: src/app.ts:3-19, src/commands/join.ts:59-71

Speech-to-Text Service (`stt`)

The stt service is a lightweight, high-performance web server written in Rust.

Framework: Built with actix-web.
Core Logic: Utilizes the whisper-rs library to run the Whisper speech recognition model. The model is loaded into memory on startup from the path specified by the MODEL_PATH environment variable.
API: It exposes a single endpoint:
- POST /: Accepts audio/wav data in the request body.
- It processes the audio, performs transcription (with translation to English enabled), and returns the resulting text as a plain string.
Performance: It is configured to use a maximum of 4 threads for processing to balance performance and resource usage.

Sources: stt/src/main.rs, stt/Cargo.toml:8-14

Translation Service (`libretranslate`)

The system uses a standard libretranslate Docker image for machine translation.

Functionality: The bot sends text to this service to be translated from English into a user's selected target language.
Configuration: The service is configured via the LT_LOAD_ONLY environment variable to only load specific language models (de, en, es, fr, it), saving memory.
API Interaction: The bot service communicates with it by sending a POST request to the /translate endpoint.

Sources: docker-compose.prod.yml:21-26, src/processors/translate.ts:12-21

Text-to-Speech (TTS) Component

Unlike other parts of the pipeline, TTS is not a standalone service. Instead, it is a Python script executed on-demand by the bot service.

Execution: The readText.ts module uses execa to run a Python script (src/lib/tts/read_text.py).
Functionality: The script takes a text string, a TTS model name, and an output file path as arguments. It uses the Coqui TTS (TTS) library to synthesize speech and save it as a .wav file.
Models: The specific TTS model used depends on the target language selected by the user, as defined in src/languages.ts.

Sources: src/processors/readText.ts:12-20, src/languages.ts:4-25

End-to-End Translation Flow

The following sequence diagram illustrates the interactions between the components.

Generating architecture diagram...

Sources: src/commands/join.ts:110-126, src/processors/recordAudio.ts, src/processors/recognizeRecording.ts, src/processors/translate.ts, src/processors/readText.ts

Audio Capture: When a user with translation enabled speaks, the speaking event listener on the VoiceReceiver is triggered in joinCommandHandler.
Recording: The recordAudio processor is called. It subscribes to the user's audio stream, decodes the Opus packets into raw PCM data using OpusScript, and writes the data to a temporary .wav file. The recording automatically stops after a brief period of silence.
Speech-to-Text: The recognizeRecording processor reads the temporary .wav file and sends its contents in a POST request to the stt service. The service transcribes the audio to English text and returns it.
Translation: The joinCommandHandler checks the user's target language. If it is not English, the translate processor sends the transcribed text to the libretranslate service, which returns the translated text.
Text-to-Speech: The readText processor is called with the final text. It executes the read_text.py script, which generates a new .wav file containing the spoken version of the text.
Playback: The path to the newly generated audio file is added to the audioQueue, which plays it back into the voice channel for all members to hear.

Sources: src/commands/join.ts:110-126, src/processors/recordAudio.ts:29-41, src/processors/recognizeRecording.ts:8-17, src/processors/translate.ts:8-26, src/processors/readText.ts:12-20

Configuration

The system is configured using environment variables, which are parsed and validated in src/env.ts using @t3-oss/env-core and zod.

Variable	Type	Optional	Default	Description
`BOT_TOKEN`	`string`	No		The Discord bot token.
`CLIENT_ID`	`string`	Yes		The Discord application client ID.
`GUILD_ID`	`string`	Yes		If set, slash commands are deployed to this guild on startup.
`LOG_PATH`	`string`	Yes		If set, logs are appended to this file path.
`POSTHOG_API_KEY`	`string`	Yes		API key for PostHog telemetry.
`POSTHOG_HOST`	`string`	Yes	`https://app.posthog.com`	Host for PostHog telemetry.
`REDIS_URL`	`string`	Yes		URL for the Redis instance.
`STT_HOST`	`string`	No		Host URL for the Speech-to-Text service.
`TRANSLATION_HOST`	`string`	No		Host URL for the LibreTranslate service.
`HEARTBEAT_URL`	`string` (URL)	No		A URL to send a POST request to every minute as a heartbeat.

Sources: src/env.ts:9-21, docker-compose.prod.yml:9-12

Slash Commands

User interaction with the bot is handled exclusively through Discord slash commands.

Command Registration

If GUILD_ID is provided, commands are registered for that specific guild on bot startup.
A separate pnpm run deploy script (which runs src/deploy.ts) can be used to register commands globally.

Sources: src/interactions.ts, src/app.ts:51-57, src/deploy.ts

Available Commands

Command	Description
`/join`	Makes the bot join the user's current voice channel.
`/leave`	Makes the bot leave the voice channel.
`/start <target>`	Starts translation for the user, setting their target language.
`/stop`	Deactivates translation for the user.
`/status`	Shows the user's current translation status and target language.
`/languages`	Shows a list of all available target languages.

Sources: README.md:14-22, src/interactions.ts:12-18

Overall System Architecture

System Components

Main Bot Application (bot)

Speech-to-Text Service (stt)

Translation Service (libretranslate)

Text-to-Speech (TTS) Component

End-to-End Translation Flow

Configuration

Slash Commands

Command Registration

Available Commands

Overall System Architecture

System Components

Main Bot Application (bot)

Speech-to-Text Service (stt)

Translation Service (libretranslate)

Text-to-Speech (TTS) Component

End-to-End Translation Flow

Configuration

Slash Commands

Command Registration

Available Commands

Main Bot Application (`bot`)

Speech-to-Text Service (`stt`)

Translation Service (`libretranslate`)

Main Bot Application (`bot`)

Speech-to-Text Service (`stt`)

Translation Service (`libretranslate`)