Quick Start Guide

Relevant source files

The following files were used as context for generating this wiki page: - [README.md](https://github.com/jackccrawford/reachy-mini-mcp/blob/HEAD/README.md) - [src/server.py](https://github.com/jackccrawford/reachy-mini-mcp/blob/HEAD/src/server.py) - [pyproject.toml](https://github.com/jackccrawford/reachy-mini-mcp/blob/HEAD/pyproject.toml) - [src/__init__.py](https://github.com/jackccrawford/reachy-mini-mcp/blob/HEAD/src/__init__.py) - [LICENSE](https://github.com/jackccrawford/reachy-mini-mcp/blob/HEAD/LICENSE)

This guide provides instructions for setting up and running the Reachy Mini MCP server, which allows AI systems like Claude, GPT, or Grok to control a Pollen Robotics' Reachy Mini robot. The server exposes a set of tools for speech, listening, vision, and emotional expression through physical movement. This project aims to give AI a physical body with zero robotics expertise required.

The system is designed with a simple, token-efficient toolset that fits within an AI's working memory. It can operate with either a physical robot or a MuJoCo-based simulator. The project is licensed under the MIT License.

Sources: README.md:3-9, src/server.py:3-17, src/init.py, LICENSE

Architecture

The system operates through a clear, linear chain of command, starting from the AI system and ending with the robot's physical or simulated actions. The MCP server acts as the central bridge, translating high-level tool calls from the AI into specific commands for the Reachy Mini SDK.

Sources: README.md:52-55

The data flow is as follows:

This diagram illustrates the control flow from the AI to the robot.
Sources: README.md:52-55, src/server.py:5-6

System Requirements

To run the Reachy Mini MCP server, the following components are required:

• Python: Version 3.10 or higher.
• Dependencies: The project uses Poetry for dependency management. Key dependencies include fastmcp, reachy-mini, openai, and websockets.
• Simulation: MuJoCo is required for running the robot simulator. On macOS, mjpython is needed for visualization.
• API Keys: A Deepgram API key is required for speech-to-text (listen) and as a fallback for text-to-speech (speak). An optional XAI_API_KEY can be used for higher-quality Grok Voice TTS.

Sources: pyproject.toml:10-14, README.md:104-108

Installation and Setup

Follow these steps to get the server running.

1. Install Dependencies:
   Clone the repository and use Poetry to install the required packages.

   cd reachy-mini-mcp
   poetry install
   

   Sources: README.md:42-44, pyproject.toml:1-21

2. Set API Keys:
   Export your Deepgram API key as an environment variable. This is necessary for the speak and listen tools.

   export DEEPGRAM_API_KEY=your_key_here
   

   Optionally, set the XAI_API_KEY to use Grok Voice for TTS.

   export XAI_API_KEY=your_xai_key_here
   

   Sources: README.md:47-48, 93-96

3. Start the Simulator (Optional):
   To run without physical hardware, start the Reachy Mini daemon with the simulator.

   mjpython -m reachy_mini.daemon.app.main --sim --scene minimal
   

   Sources: README.md:50-51

4. Run the MCP Server:
   Start the main MCP server application.

   poetry run python src/server.py
   

   This can also be run using the script defined in pyproject.toml: poetry run reachy-mini-mcp.
   Sources: README.md:53-54, pyproject.toml:16

Configuration

Environment Variables

The server's behavior can be configured using several environment variables.

Variable	Required	Default	Purpose
XAI_API_KEY	No	-	Enables Grok Voice for TTS. Preferred over Deepgram.
GROK_VOICE	No	Eve	Selects the Grok voice. Options: Ara, Eve, Leo, Rex, Sal.
DEEPGRAM_API_KEY	Yes*	-	Used for STT and as a fallback for TTS.
REACHY_DAEMON_URL	No	http://localhost:8321/api	API endpoint for the Reachy daemon.

*Required if XAI_API_KEY is not set.
Sources: README.md:91-99, src/server.py:448-451, 915

Claude Integration

To integrate with Claude applications, you need to update their respective configuration files.

Claude Desktop:
Update ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "reachy-mini": {
      "command": "poetry",
      "args": ["-C", "/path/to/reachy-mini-mcp", "run", "python", "src/server.py"],
      "env": {
        "DEEPGRAM_API_KEY": "your_key_here"
      }
    }
  }
}

Sources: README.md:63-75

Claude Code:
Update ~/.claude.json:

{
  "mcpServers": {
    "reachy-mini": {
      "command": "poetry",
      "args": ["-C", "/path/to/reachy-mini-mcp", "run", "python", "src/server.py"],
      "env": {
        "DEEPGRAM_API_KEY": "your_key_here"
      }
    }
  }
}

Sources: README.md:77-89

Core MCP Tools

The server provides 7 primary tools for robot interaction, designed to be simple and memorable.

Tool	Arguments	Purpose
speak	text, listen_after=0, voice="eve"	Voice synthesis with optional choreographed gestures and subsequent listening.
listen	duration=3	Records audio and transcribes it to text using Deepgram Nova-2.
snap	-	Captures a base64-encoded JPEG image from the robot's camera.
show	emotion, move=""	Expresses a built-in emotion or plays a pre-recorded move.
look	roll, pitch, yaw, z, duration	Sets the precise orientation of the robot's head.
rest	mode="neutral"	Sets the robot to a neutral, sleep, or wake state.
discover	library="emotions"	Lists available pre-recorded moves from a specified library.

Sources: README.md:13-25, src/server.py:302-313, 353-370, 686-694, 731-738, 770-781, 930-938, 808-820

speak Tool

The speak tool vocalizes text and can choreograph movements by parsing embedded markers in the input string.

Choreography Syntax:

speak("[move:curious1] What's this? [move:surprised1] Oh wow!")

In this example, the curious1 move is executed just before "What's this?" is spoken.

Voice Provider Logic:
The tool automatically selects the TTS provider based on environment variables.

This diagram shows the TTS provider selection logic.
Sources: src/server.py:448-453

Execution Flow:
The speak function parses the text for [move:...] markers, triggers the corresponding moves, generates audio for the text segments, and plays them. If listen_after is set, it will then activate the microphone to capture a response.

This sequence shows a choreographed speech call followed by listening.
Sources: README.md:28-31, src/server.py:686-764

show Tool

The show tool is used for physical expression. It distinguishes between 12 fast, locally-defined "emotions" and 81 more complex, pre-recorded "moves" from Pollen Robotics.

Built-in Emotions:
These are defined in the EXPRESSIONS dictionary and map an emotion name to head and antenna motor positions.

• neutral, curious, uncertain, recognition, joy, thinking
• listening, agreeing, disagreeing, sleepy, surprised, focused

Sources: src/server.py:60-158, 302-313, README.md:34-36

Recorded Moves:
These are fetched from HuggingFace libraries via the Reachy daemon. The discover() tool can be used to list all available moves, such as loving1, fear1, or serenity1.

Sources: README.md:38-42, 930-946

listen and snap Tools

• listen(duration): This tool captures audio for a set duration using the robot's microphones. The raw audio is sent to the Deepgram API for transcription.
• snap(): This captures a single frame from the robot's camera, encodes it as a JPEG, and returns it as a base64 string.

Sources: src/server.py:731-762, 808-828

Connection Management

The connection to the robot is managed via a lazy-initialized singleton pattern. The get_robot() function establishes a connection to the ReachyMini SDK on the first tool call that requires it. This avoids connecting prematurely and handles potential connection errors. A cleanup function cleanup_robot() is registered with atexit to ensure the connection is properly closed on shutdown.

This diagram shows the lazy initialization flow for the robot connection.
Sources: src/server.py:165-191, 981-983

Summary

The Reachy Mini MCP project provides a powerful yet simple interface for AI systems to interact with the physical world. By following this guide, developers can quickly set up a development environment using a simulator or a real robot, configure it for their chosen AI platform, and begin exploring embodied AI applications. The architecture is straightforward, and the toolset is intentionally concise to facilitate easy integration and use.

Sources: README.md:3-9, src/server.py:3-17