Voice Providers: Grok and Deepgram

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• src/server.py
• pyproject.toml
• src/init.py
• LICENSE

This document outlines the architecture and implementation of the voice capabilities within the Reachy Mini MCP server, which are provided by Grok for preferred Text-to-Speech (TTS) and Deepgram for fallback TTS and all Speech-to-Text (STT) functionalities. These services enable the robot to speak and listen, forming a core part of its interactive abilities.

The system is designed to be flexible, automatically prioritizing Grok's expressive voice synthesis when configured, and seamlessly falling back to Deepgram's reliable services. This dual-provider approach allows for high-quality voice interaction while maintaining operational redundancy. The primary integration points are the speak() and listen() MCP tools.

Sources: README.md:5-7, src/server.py:10-12, src/init.py:1

System Architecture

The voice system is composed of two main functions: Text-to-Speech (TTS) and Speech-to-Text (STT). TTS is handled by either Grok or Deepgram, while STT is exclusively handled by Deepgram. The selection of the TTS provider is determined by the presence of the XAI_API_KEY environment variable.

Sources: README.md:60-63, src/server.py:408-413

TTS Provider Selection Logic

The text_to_speech function acts as a router for TTS requests. It checks for the XAI_API_KEY to decide which provider's function to call.

The following diagram illustrates the provider selection flow.

Configuration

The behavior of the voice providers is controlled by environment variables.

Variable	Required	Default	Purpose
XAI_API_KEY	No	-	API key for Grok Voice TTS. Its presence makes Grok the preferred TTS provider.
GROK_VOICE	No	Eve	Specifies the Grok voice to use. Options: Ara, Eve, Leo, Rex, Sal, Mika, Valentin.
DEEPGRAM_API_KEY	Yes*	-	API key for Deepgram services. Used for STT and as a fallback for TTS.

*Required if XAI_API_KEY is not set.

Sources: README.md:92-97, src/server.py:410, src/server.py:455-457

Text-to-Speech (TTS)

TTS is initiated via the speak() tool. This tool can handle simple text as well as text with choreographed move markers. It uses the text_to_speech helper to generate audio from one of the providers.

Sources: src/server.py:590-609

Grok Voice (Preferred TTS)

When an XAI_API_KEY is provided, the system uses Grok Voice for its expressive and varied voice options. The integration uses the openai Python library, configured to communicate with xAI's API endpoint.

Sources: README.md:60-63, pyproject.toml:11, src/server.py:463-469

Available Grok Voices

The voice parameter in the speak tool or the GROK_VOICE environment variable can be set to one of the following.

Voice	Description (from speak tool docstring)
ara	Warm
eve	Energetic (default)
leo	Authoritative
rex	Confident
sal	Neutral

Note: Mika and Valentin are also listed as available voices in the README.md but are not in the speak tool's docstring.

Sources: README.md:95, src/server.py:455-457, src/server.py:608

Grok TTS Data Flow

The grok_text_to_speech function establishes a realtime connection to the Grok API, sends the text, and streams back base64-encoded audio chunks. These chunks are then decoded and assembled into a PCM data stream, which is finally saved as a temporary .wav file for playback.

This diagram shows the sequence of events for a Grok TTS request.

Deepgram (Fallback TTS)

If the XAI_API_KEY is not set, the system falls back to using Deepgram's Aura-2 model for TTS. This ensures that the robot can always speak, provided a DEEPGRAM_API_KEY is available.

Sources: README.md:60-63, src/server.py:412, src/server.py:420-422

Deepgram TTS Data Flow

The deepgram_text_to_speech function makes a single HTTP POST request to the Deepgram API with the text to be synthesized. The API returns the complete audio as an MP3 file, which is then saved to a temporary file for playback.

This diagram shows the sequence for a Deepgram TTS request.

Speech-to-Text (STT)

All STT functionality is handled by Deepgram's Nova-2 model. This service is invoked by the listen() MCP tool or as part of a speak() call if the listen_after parameter is set. The robot's microphone captures audio, which is then sent to the Deepgram API for transcription.

Sources: README.md:12, src/server.py:535-546

Deepgram STT Data Flow

The _do_listen internal function orchestrates the STT process. It records audio from the robot's microphone for a specified duration, converts the raw audio data into a WAV format in memory, and sends these bytes to the Deepgram /v1/listen endpoint. The API returns a JSON object containing the transcript.

This diagram illustrates the STT process.

Summary

The reachy-mini-mcp project employs a robust, dual-provider strategy for its voice features. It leverages Grok for high-quality, expressive TTS when available, with Deepgram serving as a reliable fallback and as the exclusive provider for STT. This architecture is configured via simple environment variables, making it easy to switch between providers. The core logic is encapsulated within the speak and listen tools, providing a simple yet powerful interface for AI systems to give the Reachy Mini robot a voice. The project is licensed under the MIT License.

Sources: README.md:60-63, LICENSE