Speech-to-Text (STT) Service (Rust)

Relevant source files

The following files were used as context for generating this wiki page:

• stt/src/main.rs
• stt/Cargo.toml
• src/processors/recognizeRecording.ts
• docker-compose.prod.yml
• src/processors/recordAudio.ts
• src/commands/join.ts

The Speech-to-Text (STT) service is a self-contained microservice written in Rust. Its sole purpose is to receive an audio file in WAV format, transcribe it into English text, and return the result. It leverages the whisper-rs library, which provides Rust bindings for the OpenAI Whisper model. This service is a critical component in the bot's translation pipeline, converting a user's spoken words into text that can then be processed and translated.

The service is designed as a simple, high-performance HTTP server built with actix-web. It exposes a single endpoint for audio processing. This architectural choice decouples the CPU-intensive task of speech recognition from the main Node.js application, allowing for independent scaling and resource management.

Architecture

The STT service is containerized and runs alongside the main bot application, as defined in the Docker Compose configuration. The main bot communicates with it over the internal Docker network.

Sources: docker-compose.prod.yml:12-14, stt/src/main.rs

Service Interaction

The following diagram illustrates how the main Node.js bot service interacts with the Rust STT service.

This diagram shows the flow from receiving a voice stream in Discord to getting transcribed text from the STT service.
Sources: src/commands/join.ts:86-92, src/processors/recognizeRecording.ts:10-16, docker-compose.prod.yml:13-14

Key Dependencies

The service relies on several key Rust crates to function.

Crate	Version	Purpose
actix-web	4	High-performance HTTP server framework.
whisper-rs	0.10.0	Rust bindings for the OpenAI Whisper model for transcription.
hound	3.5.1	A library for reading and writing WAV audio files.
lazy_static	1.4.0	Lazily evaluates a static variable, used to load the Whisper model once.
num_cpus	1.16.0	Used to determine the number of CPU cores for multithreading.

Sources: stt/Cargo.toml:7-14

API Endpoint

The service exposes a single API endpoint for processing audio.

POST /

This endpoint accepts a raw WAV audio file, processes it, and returns the transcribed text.

Aspect	Description
Method	POST
Path	/
Request	The request body must be a binary stream with the Content-Type header set to audio/wav.
Response	A plain text string (text/plain) containing the transcribed and translated English text.
On Failure	If the transcription fails, the Node.js client receives a non-ok HTTP status and returns an empty string.

Sources: stt/src/main.rs:78, src/processors/recognizeRecording.ts:12-14, src/processors/recognizeRecording.ts:18-21

Core Transcription Logic

The entire transcription process is handled by the upload async function, which is registered as the handler for the POST / route.

Sources: stt/src/main.rs:27-66, stt/src/main.rs:78

Processing Flow

The following diagram details the steps taken within the upload function to process an incoming audio file.

This flowchart shows the step-by-step process from receiving the audio to returning the final text.
Sources: stt/src/main.rs:28-65

Whisper Model Initialization and Configuration

The Whisper model is loaded into a lazy_static variable named WHISPER. This ensures the model, which can be large, is loaded from disk only once when it's first accessed, and then lives for the duration of the program. The path to the model files is specified via the MODEL_PATH environment variable.

Sources: stt/src/main.rs:13-20, stt/src/main.rs:73-76

During each transcription request, a FullParams object is created to configure the model's behavior for that specific run. The following parameters are set:

Parameter	Value	Description
SamplingStrategy	Greedy { best_of: 1 }	Uses a simple greedy decoding strategy.
n_threads	min(4, num_cpus)	Sets the number of threads to use, capped at 4.
translate	true	Crucially, this forces the model to translate the audio into English.
language	Some("en")	Sets the target language for translation to English.
print_special	false	Disables printing of special tokens (e.g., `<
print_progress	false	Disables printing transcription progress to the console.
print_realtime	false	Disables real-time printing of results.
print_timestamps	false	Disables printing of timestamps for segments.

Sources: stt/src/main.rs:41-49

Integration with the Node.js Application

The Node.js part of the application is responsible for capturing user audio, formatting it, and sending it to the STT service.

Audio Recording and Processing Sequence

The following sequence diagram shows the complete interaction, starting from when a user begins speaking in a Discord voice channel.

This diagram illustrates the full lifecycle of a single utterance from voice to text.
Sources: src/commands/join.ts:86-92, src/processors/recordAudio.ts:30-44, src/processors/recognizeRecording.ts:8-16

The recognizeRecording function in the Node.js application handles the direct communication with the STT service. It reads the temporary WAV file created by recordAudio and sends its contents in a fetch request. After receiving the response, it deletes the temporary audio file.

// src/processors/recognizeRecording.ts

const recognizeRecording = async (fileName: string) => {
  // ...
  const audio = await fs.readFile(fileName);

  const response = await fetch(env.STT_HOST, {
    method: 'POST',
    headers: {
      'Content-Type': 'audio/wav'
    },
    body: audio
  });

  if (!response.ok) {
    // ...
    return '';
  }

  const result = await response.text();

  fs.unlink(fileName) // ...

  return result;
};

Sources: src/processors/recognizeRecording.ts:8-28

Summary

The Rust STT service is a specialized, high-performance component dedicated to audio transcription. By offloading this heavy task from the main Node.js application, the system benefits from the performance of Rust and the efficiency of the Whisper C++ backend. Its simple HTTP interface and containerized deployment make it a well-integrated and essential part of the live translation pipeline, consistently providing the English text foundation upon which all subsequent translations are built.

Sources: stt/src/main.rs, docker-compose.prod.yml