Understanding the MCP Protocol
Relevant source files
The following files were used as context for generating this wiki page:
The reachy-mini-mcp project provides a server that implements the Model-Context-Protocol (MCP). This allows AI systems like Claude, GPT, or Grok to control a Pollen Robotics' Reachy Mini robot. The protocol exposes a set of seven distinct tools that enable the AI to make the robot speak, listen, see, and express emotions through physical movement. The goal is to give an AI a physical body for expressive interaction.
This server acts as a bridge, translating high-level tool calls from an AI into specific commands for the Reachy Mini SDK. It is designed to be token-efficient and easy for an AI to use, following Miller's Law by providing seven tools that can fit within a cognitive working memory.
Sources: README.md, src/server.py:1-20, src/init.py
Architecture
The system architecture is a linear pipeline that connects the AI to the physical robot or its simulator. An AI system makes a tool call, which is received by the MCP Server. The server then uses the reachy-mini SDK to communicate with the Reachy Mini Daemon, which in turn controls the robot hardware or the MuJoCo simulator.
Sources: README.md:40-43
The following diagram illustrates the data flow from the AI to the robot.
This diagram shows the high-level architecture of the system.
Sources: README.md:40-43, src/server.py:4-7
MCP Server Initialization
The core of the server is an instance of FastMCP, a library for creating MCP servers. The server is initialized with the name reachy-mini and a set of instructions for the AI model. These instructions guide the AI on how to use the available tools for robot control, emphasizing the use of show() for emotions and speak() for choreographed speech.
Sources: src/server.py:23-44
# src/server.py:23-44
mcp = FastMCP(
name="reachy-mini",
instructions="""
Reachy Mini robot control for expressive robotics.
Use these tools for robot control:
- show() for 12 built-in emotions (curious, joy, thinking, etc.)
- show(move=...) for 81 recorded emotions from Pollen (fear1, rage1, serenity1, etc.)
- discover() to see available recorded moves
- look() for precise head positioning
- speak() to vocalize with [move:X] markers for choreography
- listen() to hear and transcribe speech
- snap() to capture camera images
- rest() for neutral pose, sleep, or wake
Prefer show() for common emotions, show(move=...) for nuanced expressions.
"""
)
Sources: src/server.py:23-44
MCP Tools
The server exposes seven tools for robot control, which are defined as functions decorated with @mcp.tool().
Sources: README.md:13-25
| Tool | Args | Purpose |
|---|---|---|
speak | text, listen_after=0 | Voice + gesture, optionally listen after |
listen | duration=3 | STT via Deepgram Nova-2 |
snap | - | Camera capture (base64 JPEG) |
show | emotion, move="" | Express emotion or play recorded move |
look | roll, pitch, yaw, z, duration | Head positioning (degrees) |
rest | mode="neutral" | neutral / sleep / wake |
discover | library="emotions" | Find available recorded moves |
Sources: README.md:13-25
show()
This tool allows the robot to express an emotion through physical movement. It has two modes of operation:
- Built-in Emotions: Using the
emotionparameter, one of 12 pre-defined, locally-executed emotions can be triggered. - Recorded Moves: Using the
moveparameter, one of 81 professionally choreographed moves from Pollen Robotics' library can be played. Themoveparameter takes precedence overemotion.
Sources: src/server.py:313-345, README.md:29-37
The following diagram shows the logic flow for the show tool.
Logic flow for the show tool, deciding between a recorded move and a built-in emotion.
Sources: src/server.py:343-345
The 12 built-in emotions are defined in the EXPRESSIONS dictionary, which maps an emotion name to head position, antenna angles, movement duration, and an interpolation method.
Sources: src/server.py:52-161
speak()
The speak tool vocalizes text through the robot's speaker. It supports choreographed speech by parsing embedded [move:name] markers within the text. When a marker is found, the corresponding move is triggered just before the subsequent text chunk is spoken.
The tool can also listen for a response for a specified duration after speaking is complete.
Sources: src/server.py:613-698, README.md:27-28
The following sequence diagram illustrates a choreographed speech call.
Sequence of events for a speak call with an embedded move.
Sources: src/server.py:651-683
listen()
This tool captures audio from the robot's microphones for a specified duration and transcribes it to text using Deepgram's Nova-2 STT service. The internal helper _do_listen handles the audio recording and conversion to WAV format before sending it for transcription.
Sources: src/server.py:778-794, src/server.py:701-748
snap()
Captures a single frame from the robot's camera, encodes it as a JPEG, and returns it as a base64-encoded string. This allows the AI to "see" its environment.
Sources: src/server.py:797-820
look()
Provides direct, precise control over the robot's head orientation (roll, pitch, yaw), vertical position (z), and movement duration. Input values are clamped to safe ranges to prevent mechanical damage.
Sources: src/server.py:348-390
rest()
Controls the robot's power state.
neutral: Returns the robot to a default, upright pose.sleep: Puts the robot into a low-power rest state.wake: Wakes the robot from sleep mode.
Sources: src/server.py:823-844
discover()
Queries the Reachy Mini Daemon to find available recorded moves from Pollen Robotics' HuggingFace libraries. This allows the AI to learn which moves can be used with show(move=...).
Sources: src/server.py:877-901
Core Components
Robot Connection Management
Connection to the robot is handled via a lazy-initialized singleton pattern. The get_robot() function establishes a connection to the ReachyMini daemon on the first tool call that requires it. This avoids connecting at server startup. The cleanup_robot() function is registered with atexit to ensure the connection is properly closed on shutdown.
Sources: src/server.py:167-199
Lazy initialization flow for the robot connection.
Sources: src/server.py:173-190
Voice Services
The server supports two Text-to-Speech (TTS) providers:
- Grok Voice: Preferred provider, used if the
XAI_API_KEYenvironment variable is set. - Deepgram Aura 2: Fallback provider, used if Grok is not configured. Requires
DEEPGRAM_API_KEY.
Speech-to-Text (STT) is handled exclusively by Deepgram Nova-2.
Sources: README.md:45-51, src/server.py:393-400, src/server.py:541-564
Logic for selecting the TTS provider.
Sources: src/server.py:397-400
Configuration
Environment Variables
The server's behavior, particularly for voice services, is configured via environment variables.
| Variable | Required | Default | Purpose |
|---|---|---|---|
XAI_API_KEY | No | - | Grok Voice TTS (preferred) |
GROK_VOICE | No | Eve | Grok voice: Ara, Eve, Leo, Rex, Sal, Mika, Valentin |
DEEPGRAM_API_KEY | Yes* | - | TTS (fallback) + STT |
REACHY_DAEMON_URL | No | http://localhost:8321/api | Daemon API endpoint |
Required if XAI_API_KEY is not set.
Sources: README.md:78-85
Claude Configuration
To integrate with Claude, the MCP server must be declared in the Claude configuration file. The command and args specify how to launch the server using poetry.
Claude Desktop (claude_desktop_config.json):
{
"mcpServers": {
"reachy-mini": {
"command": "poetry",
"args": ["-C", "/path/to/reachy-mini-mcp", "run", "python", "src/server.py"],
"env": {
"DEEPGRAM_API_KEY": "your_key_here"
}
}
}
}
Sources: README.md:57-69
Dependencies
The project relies on several key Python libraries, defined in pyproject.toml.
fastmcp: For the underlying MCP server implementation.reachy-mini: The official SDK for controlling the robot.openai: Used as a client for the Grok Voice API.websockets: A dependency forfastmcp.
Sources: pyproject.toml:10-14
Summary
The reachy-mini-mcp project provides a robust and well-defined MCP implementation for controlling the Reachy Mini robot. By exposing a concise set of seven tools, it creates a powerful yet simple interface for advanced AI models to interact with the physical world. The architecture effectively abstracts the complexities of robotics, TTS/STT services, and motion choreography, allowing the AI to focus on high-level expressive tasks. The support for choreographed speech and multiple emotion libraries makes it a versatile platform for embodied AI experimentation.
Sources: README.md, src/server.py