Architecture Overview

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• pyproject.toml
• src/server.py
• src/init.py
• LICENSE

The reachy-mini-mcp project provides a Model-Context-Protocol (MCP) server that acts as a bridge between AI systems (like Claude, GPT, or Grok) and Pollen Robotics' Reachy Mini robot. This enables an AI to control the robot's physical actions, including speech, movement, emotional expression, and sensory input (vision and hearing). The architecture is designed to be simple and efficient, exposing a limited set of seven high-level tools to the AI, following Miller's Law for cognitive ease.

The system is composed of several key layers: the AI system making tool calls, the MCP server translating those calls, the Reachy Mini SDK providing a Python API for the robot, a daemon process managing the low-level hardware communication, and finally the physical robot or its simulator. This layered approach abstracts the complexities of robotics, allowing AI developers to imbue their systems with a physical body without requiring specialized robotics expertise.

Sources: README.md:3-7, src/server.py:5-16

High-Level System Flow

The core architectural flow is a unidirectional chain of command from the AI to the robot. An AI system invokes a tool, which is processed by the MCP server, passed to the robot's SDK, and executed by the daemon on the hardware or in a simulator.

This diagram illustrates the primary data flow between components.

Components

MCP Server (src/server.py)

The heart of the project is the server.py file, which implements the MCP server using the fastmcp library. It defines the seven tools available to the AI and contains the logic for translating these high-level commands into specific actions using the Reachy Mini SDK.

Key Responsibilities:

• Initialize the FastMCP server with a name (reachy-mini) and instructions for the AI.
• Define and expose 7 tools using the @mcp.tool() decorator.
• Manage the connection to the robot via a lazy-initialized ReachyMini instance.
• Handle text-to-speech (TTS) and speech-to-text (STT) operations through external voice providers.
• Parse and execute choreographed speech and movement.

Sources: src/server.py:20-41, pyproject.toml:13

MCP Tools

The server exposes seven tools designed to be a concise but powerful interface for robot control.

Tool	Arguments	Purpose	Source File
speak	text, listen_after=0, voice="eve"	Voice + gesture, with optional listening after. Supports choreographed moves.	src/server.py:614-679
listen	duration=3	Speech-to-text (STT) via Deepgram Nova-2.	src/server.py:768-785
snap	-	Captures a base64-encoded JPEG image from the robot's camera.	src/server.py:788-810
show	emotion, move=""	Expresses a built-in emotion or plays a pre-recorded move.	src/server.py:316-351
look	roll, pitch, yaw, z, duration	Sets precise head position in degrees.	src/server.py:354-394
rest	mode="neutral"	Sets the robot to a neutral, sleep, or wake state.	src/server.py:813-833
discover	library="emotions"	Lists available recorded moves from Pollen's libraries.	src/server.py:869-894

Sources: README.md:11-23, src/server.py

Robot Connection Management

Connection to the robot is not established on server startup but is created on-demand when the first tool is called. The get_robot() function implements this lazy initialization pattern. It creates and caches a global _robot_instance of the ReachyMini class. A cleanup function cleanup_robot() is registered with atexit to ensure the connection is properly closed on server shutdown.

This diagram shows the lazy initialization flow.

Voice and Speech Processing

The system integrates with external services for Text-to-Speech (TTS) and Speech-to-Text (STT).

TTS Provider Logic

The text_to_speech function dynamically chooses a provider. It prioritizes Grok Voice if the XAI_API_KEY environment variable is set. Otherwise, it falls back to Deepgram's Aura-2 model, which requires the DEEPGRAM_API_KEY.

Choreographed Speech (speak tool)

The speak tool has advanced capabilities for choreographing movement with speech. It parses the input text for markers like [move:curious1].

The execution flow is as follows:

1. The _parse_choreographed_text helper function splits the text into a sequence of text and move segments.
2. The speak function iterates through these segments.
3. When a move segment is encountered, it is queued.
4. When a text segment is encountered, the queued move is executed first. The system then waits for the move to complete.
5. After the move is done, the text segment is converted to audio and played.
6. This process repeats for all segments.

This sequence diagram shows an example call to speak("Hello [move:joy] world").

Expression and Movement

The robot's physical expressions are handled by two main mechanisms: built-in emotions and recorded moves.

Built-in Emotions (show(emotion=...))

The src/server.py file defines a dictionary named EXPRESSIONS that maps 12 high-level emotion strings to specific motor coordinates for the head and antennas, along with movement duration and interpolation style. These are executed locally by the _do_express function, which calls the SDK's goto_target method.

Sources: src/server.py:51-166, src/server.py:280-307

Recorded Moves (show(move=...))

For more complex or nuanced movements, the system can play pre-recorded moves hosted on HuggingFace. The discover tool queries the daemon to list available moves from libraries like pollen-robotics/reachy-mini-emotions-library. The show(move=...) command triggers a POST request to the daemon's /move/play/... endpoint to execute the specified move.

Sources: src/server.py:859-864, src/server.py:869-894, src/server.py:897-917

Configuration

Configuration is managed primarily through environment variables.

Variable	Required	Default	Purpose
XAI_API_KEY	No	-	API key for Grok Voice TTS (preferred).
GROK_VOICE	No	Eve	Specifies the Grok voice to use (e.g., Ara, Eve, Leo).
DEEPGRAM_API_KEY	Yes*	-	API key for Deepgram TTS (fallback) and STT.
REACHY_DAEMON_URL	No	http://localhost:8321/api	The API endpoint for the Reachy daemon.

*Required if XAI_API_KEY is not set.

Sources: README.md:100-105, src/server.py:402, src/server.py:856

Dependencies

The project relies on several key Python packages, which are managed by Poetry.

Package	Version	Purpose
fastmcp	^2.0.0	Core library for creating the MCP server.
reachy-mini	develop branch	The official SDK for controlling the robot.
openai	^1.50.0	Used as a client for the Grok Voice API.
websockets	^13.0	Dependency for real-time communication (likely for Grok Voice).

Sources: pyproject.toml:12-16