Muse App Integration Flow
Relevant source files
The following files were used as context for generating this wiki page:
The Muse App Integration Flow establishes a secure communication channel between an ESP32-based gadget and the Muse mobile application. This process encompasses Bluetooth Low Energy (BLE) discovery, Wi-Fi provisioning, and the establishment of an encrypted Noise session to a Virtual Machine (VM) hosted by Muse.
The integration enables the Muse app to control hardware features, perform over-the-air (OTA) updates, and facilitate voice-based interactions. The firmware manages hardware-specific UI states, such as LED status indicators and on-screen animations, to provide feedback during the various stages of the integration process.
Sources: board-port/esp32/README.md:143-162, board-port/esp32/AGENTS.md:32-37
Integration Lifecycle
The integration flow moves through distinct states, primarily driven by the app.c module and communicated to the user via the led_status.c module.
Setup and Provisioning States
When a device is unprovisioned, it enters an advertising state to wait for the Muse app. The following table describes the visual indicators for these states:
| State | Visual Indicator (LED/Screen) | Description |
|---|---|---|
| Ready for Setup | Orange, breathing | Device is advertising over BLE and waiting for the Muse app. |
| Confirm Pairing | Blue, breathing | A pairing request is received; requires a physical button press. |
| Connecting | Blue, solid | Device is joining Wi-Fi and establishing the Noise session. |
| Connected | Green, solid | Successfully connected to the Muse VM and tunnel is active. |
| Error | Red, blinking | An error occurred during setup or operation. |
Sources: board-port/esp32/README.md:164-173, board-port/esp32/main/app.c:165-185, board-port/esp32/main/led_status.c:980-1015
Provisioning Sequence
The sequence below illustrates the communication between the Muse App and the ESP32 Device.
The sequence begins with BLE advertising, followed by the transmission of Wi-Fi credentials and SDK tokens from the app to the device. A physical button press on the device is mandatory to prevent unauthorized pairing.
Sources: board-port/esp32/main/app.c:661-750, board-port/esp32/main/app.c:1049-1065, board-port/esp32/AGENTS.md:386-395
Communication Architecture
Integration relies on two primary communication layers: BLE for initial setup and an encrypted Noise protocol over WebSockets for ongoing operation.
Noise Control Session
The noise_control.cpp module manages the secure connection to the Muse backend. It establishes a WebSocket connection to hatch.metaaivm.com (default) and performs a Noise handshake.
Sources: board-port/esp32/main/noise_control.cpp:66-70, board-port/esp32/main/noise_control.cpp:525-560
Key attributes of the Noise session include:
- Stream Multiplexing: Different data types use specific stream IDs (e.g., ID 1 for Control, ID 2 for Tunnel, ID 3 for Identity).
- Heartbeats: The device sends
link.heartbeatmessages every 60 seconds to maintain the connection. - Command Handling: The session task dispatches JSON-RPC style commands from Muse to local handlers in
app.c.
The diagram shows how the Noise Control Task demultiplexes incoming data based on Stream IDs to appropriate internal modules.
Sources: board-port/esp32/main/noise_control.cpp:71-77, board-port/esp32/main/noise_control.cpp:1135-1160
Integration Commands
Once paired, the Muse app interacts with the device using a registered set of commands. These are advertised by the device during the link.register phase.
| Command | Purpose | Parameters |
|---|---|---|
device.ota | Updates device firmware | url (string), force (bool) |
display.draw_url | Renders an image on screen | url (string), row (int) |
device.health | Reports system metrics | None |
voice.configure | Sets audio parameters | volume (int) |
sensors.read | Retrieves sensor data | None |
Sources: board-port/esp32/AGENTS.md:522-540, board-port/esp32/main/noise_control.cpp:815-920
Handling Commands in app.c
Significant logic for command execution resides in on_ws_command. If a command requires long-running operations (like device.discover or camera.capture), it is executed asynchronously in a separate task to avoid blocking the Noise session task.
Sources: board-port/esp32/main/app.c:1152-1210, board-port/esp32/AGENTS.md:541-558
Security and Identity
Integration security is anchored by three components:
- SDK Token: A unique identifier (
mgst_...) obtained fromgadgets.muse.aithat is baked into the firmware. - Device Identity: A
node_idderived from hardware identifiers to uniquely represent the gadget. - NVS Encryption: Recommended for storing Wi-Fi credentials and device tokens securely on the ESP32 flash.
Sources: board-port/esp32/README.md:189-195, board-port/esp32/main/app.c:1560-1580, board-port/esp32/AGENTS.md:200-205
Conclusion
The Muse App Integration Flow provides a structured approach for connecting community-built hardware to the Muse ecosystem. By combining BLE-based provisioning with a robust, encrypted Noise session, the system ensures both ease of use for setup and high security for ongoing cloud-to-edge communication. The modular design allows for diverse hardware support, from simple LED-based boards to complex devices with full graphical UIs and audio capabilities.