Tool: snap()

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• src/server.py
• pyproject.toml
• src/init.py
• LICENSE

The snap() tool is a core function within the Reachy Mini MCP server, designed to provide visual perception capabilities to a connected AI system. It is one of the seven primary tools available for controlling the Reachy Mini robot. The function's sole purpose is to capture a single image from the robot's camera and return it in a machine-readable format. This allows an AI to "see" its environment and use that visual information for decision-making. The project is licensed under the MIT License.

Sources: README.md:10-16, src/server.py:12, 946-948, LICENSE

Function Definition

The snap() tool is defined as an MCP tool with no arguments. It is decorated with @mcp.tool() to register it with the FastMCP server.

Sources: src/server.py:945-954

@mcp.tool()
def snap() -> str:
    """
    Capture an image from the robot's camera.

    Returns the current view as base64-encoded image.
    Use this to perceive the environment.

    Returns:
        Base64-encoded image data (JPEG)
    """

Sources: src/server.py:946-954

Execution Flow

The snap() function follows a clear sequence to capture, process, and return an image. The process involves connecting to the robot, capturing a raw frame, encoding it, and formatting the output.

This sequence diagram illustrates the interactions between the components when snap() is called.

Detailed Steps

1. Get Robot Instance: The function first calls get_robot() to ensure a connection to the Reachy Mini is established. This uses lazy initialization, connecting only on the first tool call.
   Sources: src/server.py:183-186, 955
2. Capture Frame: It then invokes robot.media.get_frame() via the Reachy Mini SDK to retrieve the current camera view as a raw image frame (typically a NumPy array).
   Sources: src/server.py:958
3. Encode to JPEG: If a frame is successfully captured, the cv2.imencode function is used to encode the raw frame into the JPEG format. This requires the OpenCV library to be available.
   Sources: src/server.py:961
4. Encode to Base64: The resulting JPEG image buffer is then encoded into a base64 string. This makes the binary image data safe for transport as a JSON-compatible string.
   Sources: src/server.py:962
5. Format Output: The final base64 string is prepended with data:image/jpeg;base64, to form a complete data URI, which is a standard way to embed images in web documents and other text-based formats.
   Sources: src/server.py:963

Return Value

The function returns a single string. The structure of this string is a data URI, which is detailed below.

Part	Description	Example
Scheme	Identifies the content as embedded data.	data:
MIME Type	Specifies the media type of the data. For snap(), this is always JPEG.	image/jpeg
Encoding	Specifies the encoding used for the data.	base64
Data	The base64-encoded image data.	iVBORw0KGgoAAAANSUhEUg...

A successful call returns the full data URI string. If no frame is captured, it returns the string "No frame captured".
Sources: src/server.py:963, 965

Error Handling

The snap() function includes basic error handling for two primary failure modes:

• OpenCV Not Found: If the cv2 library (OpenCV) cannot be imported, an ImportError is caught, and the function returns the string "OpenCV not available for image encoding".
  Sources: src/server.py:967-968
• General Vision Failure: Any other exception during the process (e.g., failure to get a frame from the daemon) is caught, and a formatted error message f"Vision failed: {e}" is returned.
  Sources: src/server.py:969-970

Dependencies

The snap() tool's functionality relies on several key dependencies.

Dependency	Purpose	Source
reachy-mini SDK	Provides the robot.media.get_frame() method to interface with the robot's hardware/simulator.	pyproject.toml:12
fastmcp	The framework used to define snap() as a callable tool.	pyproject.toml:11
opencv-python	Used via import cv2 for encoding the raw image frame into JPEG format. This is a runtime dependency handled within the code.	src/server.py:960, 967
base64	A standard Python library used for encoding the image data.	src/server.py:25, 962

Sources: pyproject.toml:10-12, src/server.py:24, 960-968