Audio Recording

Relevant source files

The following files were used as context for generating this wiki page:

• src/processors/recordAudio.ts
• src/commands/join.ts
• src/processors/recognizeRecording.ts
• stt/src/main.rs
• package.json
• src/utils/AudioQueue.ts
• src/processors/readText.ts

The audio recording system is a fundamental component of the Discord Live Translator bot. Its primary responsibility is to capture a user's speech from a Discord voice channel, process it into a usable format, and save it as a temporary audio file. This file then serves as the input for the subsequent Speech-to-Text (STT) and translation pipeline. The entire process is designed to be automatic, triggering whenever a user with translation enabled begins to speak.

The system is built upon the @discordjs/voice library for receiving audio streams from Discord. It uses a custom stream pipeline to decode the incoming Opus audio packets into raw PCM data, which is then encoded and saved as a .wav file. This ensures compatibility with the backend STT service.

Core Components and Dependencies

The audio recording functionality relies on several key Node.js packages and internal modules.

Component / Package	Description	Source
@discordjs/voice	Provides the core functionality for interacting with Discord voice, including receiving audio streams (VoiceReceiver).	package.json:20
opusscript	Used to decode the Opus audio packets received from Discord. It is chosen over @discordjs/opus for its broader platform support (e.g., macOS arm64).	package.json:26, src/processors/recordAudio.ts:5
wav	A library used to encode the decoded audio data into the .wav file format. The FileWriter class is specifically used.	package.json:29, src/processors/recordAudio.ts:7
OpusDecodingStream	A custom Transform stream class that takes Opus packets and uses an OpusScript instance to decode them into raw audio data.	src/processors/recordAudio.ts:11-21

Recording Process Flow

The process of recording a user's audio is initiated by a Discord event and involves a pipeline of streams to capture, decode, and save the audio.

Initiation

The recording process is not manually triggered but starts automatically when the bot detects a user speaking in the voice channel it is connected to.

1. A user with translation enabled speaks in the voice channel.
2. The connection.receiver from @discordjs/voice emits a speaking.on('start', ...) event, providing the userId of the speaker.
3. The event listener, defined in the joinCommandHandler, calls the recordAudio function to begin capturing the audio for that specific user.

Sources: src/commands/join.ts:85-88

Audio Capture and Decoding

The recordAudio function orchestrates the capture and processing of the audio stream. It sets up a pipeline to convert the incoming audio into a saved file.

The following diagram illustrates the stream pipeline within the recordAudio function.

1. Subscribe to Stream: The receiver.subscribe(userId) method creates a readable stream (opusStream) of Opus audio packets from the specified user. The subscription is configured to end after 100ms of silence (EndBehaviorType.AfterSilence).
   Sources: src/processors/recordAudio.ts:28-34
2. Instantiate Decoder: An OpusScript instance is created with a target sample rate of 16000 Hz and 1 audio channel (mono). This is passed to the custom OpusDecodingStream.
   Sources: src/processors/recordAudio.ts:23, 35
3. Setup File Writer: A wav.FileWriter is initialized. It is configured to create a .wav file with a 16000 Hz sample rate and 1 channel, matching the decoder's output.
   Sources: src/processors/recordAudio.ts:36-39
4. Pipeline Execution: The node:stream/promises.pipeline function connects the opusStream, the decodingStream, and the fileWriter. This efficiently directs the flow of data from the source, through the decoder, and into the final file without high memory usage.
   Sources: src/processors/recordAudio.ts:43

File Storage

The recorded audio is saved as a temporary .wav file in the local filesystem.

• Directory: Files are stored in the cache/rec/ directory relative to the project root. The join command handler ensures this directory exists before attempting to record.
  Sources: src/commands/join.ts:54
• File Naming: The filename is generated using a pattern of [userId]_[timestamp].wav to ensure uniqueness and traceability. For example: 424242424242_1677628800000.wav.
  Sources: src/processors/recordAudio.ts:25

Upon successful completion of the pipeline, the recordAudio function returns the full path to the newly created .wav file. This path is then passed to the recognizeRecording processor. If an error occurs during the pipeline, it is caught and logged, and the function returns undefined.

Sources: src/processors/recordAudio.ts:45-49

Full Translation Sequence

Audio recording is the first step in a larger sequence that involves speech-to-text, translation, and text-to-speech. The following diagram shows the complete end-to-end flow, starting from the user speaking.

This diagram illustrates how the output of the audio recording (.wav file) is consumed by the recognizeRecording function, which sends it to the Rust-based STT service. The resulting text is then translated and converted back to speech, which is finally played back in the voice channel via the AudioQueue.

Sources: src/commands/join.ts:85-101, src/processors/recordAudio.ts, src/processors/recognizeRecording.ts, stt/src/main.rs, src/processors/translate.ts, src/processors/readText.ts, src/utils/AudioQueue.ts:60-64