Data Flow

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• src/server.py
• pyproject.toml
• src/init.py
• LICENSE

The reachy-mini-mcp project acts as a bridge, translating high-level commands from an AI system into low-level actions for the Reachy Mini robot. It exposes a set of tools via the Model-Context Protocol (MCP) that an AI can call to make the robot speak, listen, see, and express emotions. The core of the data flow is a one-way street from the AI to the robot, abstracting the complexities of robotics into a simple, token-efficient toolset.

This server receives a tool call, processes it, and communicates with the underlying reachy-mini SDK and its associated daemon process. The daemon, in turn, controls the physical robot or its simulator. Data flows back to the AI primarily as the results of tool calls, such as transcribed text from listening or a base64-encoded image from the camera.

Sources: README.md, src/server.py:4-20

Overall Architecture

The system is composed of several distinct components that communicate in a linear chain. An AI system (like Claude, GPT, or Grok) initiates a tool call, which is received by the MCP Server. The server then uses the Reachy Mini SDK to send commands to the Reachy Daemon, which finally controls the robot hardware or simulator.

Sources: README.md:32-34

This diagram illustrates the high-level data flow from the AI to the robot.

MCP Server Core

The server is built using the fastmcp library. It initializes a FastMCP instance and defines seven tools for robot control. These tools are the exclusive entry points for any data or command to enter the system from the AI.

Sources: src/server.py:23-44

Tool Definitions

The server exposes 7 tools to the AI. Each tool is a Python function decorated with @mcp.tool().

Tool	Arguments	Purpose	Source File
speak	text, listen_after=0, voice="eve"	Voice synthesis with optional choreographed moves and subsequent listening.	src/server.py:610-667
listen	duration=3	Records audio and transcribes it using Deepgram Nova-2.	src/server.py:726-743
snap	-	Captures a JPEG image from the robot's camera and returns it as base64.	src/server.py:746-768
show	emotion, move=""	Expresses a built-in emotion or plays a pre-recorded move.	src/server.py:314-351
look	roll, pitch, yaw, z, duration	Sets the precise orientation of the robot's head.	src/server.py:354-398
rest	mode="neutral"	Sets the robot to a neutral, sleep, or wake state.	src/server.py:771-791
discover	library="emotions"	Lists available pre-recorded moves from a specified library.	src/server.py:821-847

Sources: README.md:12-25, src/server.py

Robot Connection Management

The connection to the robot is managed through a lazy-initialized global instance. The get_robot() function establishes a connection to the ReachyMini SDK on the first tool call that requires it. This avoids connecting to the daemon if the server is started but not used. A cleanup function, cleanup_robot(), is registered with atexit to ensure the connection is properly closed on server shutdown.

Sources: src/server.py:169-192, src/server.py:882-884

This diagram shows the lazy initialization flow.

Command Execution Data Flow

Each tool follows a specific data flow to execute its function.

Movement and Expression (show, look, rest)

Movement commands translate abstract concepts (like "curious") or specific coordinates into motor positions.

• show(emotion=...): This call is handled by the _do_express helper. It looks up the specified emotion in the EXPRESSIONS dictionary, which maps emotion names to head poses and antenna angles. It then calls robot.goto_target() with the retrieved parameters.
• show(move=...): This call is delegated to _do_play_move. It makes an HTTP POST request to the Reachy Daemon's API endpoint (/move/play/recorded-move-dataset/...) to trigger a pre-choreographed routine.
• look(...): This provides direct control over the head's position. It clamps the input values to safe ranges and calls robot.goto_target() with a transformation matrix created by create_head_pose_array.

Sources: src/server.py:288-309, src/server.py:314-351, src/server.py:354-398, src/server.py:849-870

The flow for the show tool demonstrates its dual nature.

Speech Synthesis (speak)

The speak tool is the most complex, orchestrating Text-To-Speech (TTS) and optional choreographed movements.

1. Choreography Parsing: If the input text contains [move:...] markers, the _parse_choreographed_text function splits the string into a list of text and move segments.
2. TTS Provider Selection: The text_to_speech function determines which TTS service to use. If the XAI_API_KEY environment variable is set, it uses Grok Voice; otherwise, it falls back to Deepgram.
3. Execution Loop: The server iterates through the segments. For each move segment, it calls _do_move. For each text segment, it calls the selected TTS provider, saves the audio to a temporary file, and plays it using robot.media.play_sound().
4. Listening: If listen_after is greater than 0, it waits for the speech and moves to finish, then calls _do_listen to capture and transcribe audio.

Sources: src/server.py:610-667, src/server.py:401-410, src/server.py:570-607

The sequence diagram below details the flow for a choreographed speak call.

Speech Recognition (listen)

The listen tool captures audio and converts it to text.

1. Audio Capture: The internal _do_listen function calls robot.media.start_recording(), waits for the specified duration, and retrieves the audio sample as a NumPy array.
2. Data Formatting: The raw audio data is converted into WAV format in an in-memory buffer.
3. Transcription: The speech_to_text function sends the WAV data to the Deepgram API (/v1/listen) via an HTTP POST request.
4. Response: The JSON response from Deepgram is parsed to extract the transcript, which is returned to the AI.

Sources: src/server.py:701-743, src/server.py:531-567

Vision (snap)

The snap tool captures a single frame from the robot's camera.

1. Frame Capture: It calls robot.media.get_frame() to get the current camera image as a raw frame (NumPy array).
2. Encoding: The cv2.imencode function is used to encode the frame into JPEG format.
3. Base64 Conversion: The resulting JPEG buffer is base64-encoded and formatted into a data URI string (data:image/jpeg;base64,...).
4. Response: The base64 string is returned to the AI, allowing it to "see" what the robot sees.

Sources: src/server.py:746-768

Configuration and Environment

The data flow can be altered by environment variables, which control API keys and service endpoints.

Variable	Purpose	Effect on Data Flow	Source
XAI_API_KEY	API key for Grok Voice.	If set, speak() uses Grok TTS. Otherwise, it falls back to Deepgram.	README.md:82-85, src/server.py:404-407
GROK_VOICE	Specifies the voice to use with Grok TTS.	Changes the voice used in the request to the Grok API.	README.md:82-85, src/server.py:441-445
DEEPGRAM_API_KEY	API key for Deepgram.	Required for listen() and for speak() if XAI_API_KEY is not set.	README.md:82-85, src/server.py:414-416, src/server.py:534-536
REACHY_DAEMON_URL	The URL for the Reachy Daemon API.	Determines the endpoint for playing recorded moves (show(move=...)) and discovering them (discover()).	README.md:82-85, src/server.py:796

Sources: README.md:82-89, src/server.py

Summary

The data flow in reachy-mini-mcp is a well-defined, unidirectional process that begins with a high-level command from an AI and ends with a physical action by the robot. The server acts as a crucial abstraction layer, handling connection management, command translation, and interaction with external services like TTS and STT APIs. By exposing a limited but powerful set of tools, it enables complex, expressive behavior from the robot without requiring the AI to have any knowledge of the underlying robotics hardware or software stack.

Sources: src/server.py, README.md