Running on Physical Hardware

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• pyproject.toml
• src/server.py
• src/init.py
• LICENSE

This document outlines the architecture, configuration, and operational details for running the reachy-mini-mcp server with a physical Reachy Mini robot. The server is designed to function identically for both simulation and real hardware, with connection details handled automatically. The primary goal is to provide a body for an AI, enabling it to speak, listen, see, and express emotions through physical movement.

The system allows an AI (like Claude, GPT, or Grok) to control the robot using a set of seven tools. This enables complex, choreographed interactions by abstracting away low-level robotics controls.

Sources: README.md:3-8, src/init.py:1

Architecture

The reachy-mini-mcp server acts as a bridge between a high-level AI system and the Reachy Mini robot's hardware control daemon. The AI sends tool calls to the MCP server, which translates them into commands for the Reachy Mini SDK. The SDK then communicates with the daemon, which in turn controls the physical robot or simulator.

Sources: README.md:52-55

The following diagram illustrates the data flow from the AI to the robot:

Connection Management

Connection to the robot is managed lazily, meaning the connection is only established when the first tool call is made. This is handled by the get_robot() function.

Sources: src/server.py:190-194

Robot Initialization

The get_robot() function checks if a global _robot_instance exists. If not, it imports the ReachyMini class from the SDK and attempts to establish a connection. On successful connection, the instance is stored for subsequent tool calls. If the connection fails, a RuntimeError is raised, typically indicating that the robot's daemon is not running.

The following diagram shows the connection logic:

Media Backend Configuration

For operation with a physical robot, enabling the camera and audio hardware is crucial. This is controlled by the REACHY_MEDIA_BACKEND environment variable.

• default: Use this for real hardware. It enables full media support, including audio (microphone/speaker) and the camera.
• default_no_video: (Default) Use this for simulation or when only audio is needed.
• no_media: Use for headless operation without audio or camera.

The server code explicitly reads this environment variable to configure the ReachyMini instance.

# src/server.py:197-202
            # Use 'default' for full media (audio + camera) - requires real hardware
            # Use 'default_no_video' for audio only (simulator compatible)
            # Use 'no_media' for headless (no audio or camera)
            backend = os.environ.get("REACHY_MEDIA_BACKEND", "default_no_video")
            _robot_instance = ReachyMini(media_backend=backend)

Sources: src/server.py:197-202

Configuration

Correct configuration is essential for connecting the MCP server to the robot daemon and external voice services.

Environment Variables

The server's behavior is configured using several environment variables.

Variable	Required	Default	Purpose
XAI_API_KEY	No	-	API key for Grok Voice TTS (preferred).
GROK_VOICE	No	Eve	Specifies the Grok voice to use.
DEEPGRAM_API_KEY	Yes*	-	API key for Deepgram TTS (fallback) and STT.
REACHY_DAEMON_URL	No	http://localhost:8321/api	The API endpoint for the Reachy Mini daemon.
REACHY_MEDIA_BACKEND	No	default_no_video	Sets media backend. Use default for physical hardware.

*Required if XAI_API_KEY is not set.

Sources: README.md:104-110, src/server.py:201, src/server.py:910

Daemon Connection

The MCP server communicates with the robot via the Reachy Mini daemon. By default, it expects the daemon to be running at http://localhost:8321/api. This can be overridden by setting the REACHY_DAEMON_URL environment variable. The daemon is responsible for auto-connecting to the physical hardware.

Sources: README.md:109, README.md:116, src/server.py:910

Hardware-Dependent Tools

Several MCP tools rely on the physical robot's hardware (camera, microphones, speakers) to function correctly. These require the REACHY_MEDIA_BACKEND to be set to default.

speak() and listen()

• The speak() tool uses either Grok Voice or Deepgram for Text-to-Speech (TTS) and plays the resulting audio through the robot's speaker via robot.media.play_sound().
• The listen() tool captures audio from the robot's microphones using robot.media.start_recording() and robot.media.get_audio_sample(), then sends it to Deepgram for Speech-to-Text (STT).

The sequence diagram below shows the flow for a speak command followed by listen.

snap()

The snap() tool captures a single frame from the robot's camera using robot.media.get_frame(). The image is then encoded as a base64 JPEG string and returned to the AI, allowing it to "see" its environment. This functionality is only available on physical hardware with the camera enabled.

Sources: src/server.py:813-834

Hardware Notes and Troubleshooting

• Daemon Auto-Connection: The Reachy Mini daemon is responsible for finding and connecting to the robot hardware automatically. The MCP server only needs to connect to the daemon.
• Port Conflicts: Be aware of potential port conflicts. The Zenoh protocol used by the robot stack typically uses port 7447, and the daemon defaults to port 8321.
• Dependencies: Ensure the reachy-mini SDK is installed, as specified in the project's dependencies. This provides the necessary ReachyMini class for hardware interaction.

Sources: README.md:116-118, pyproject.toml:12-13