Unpacking Artifacts
Relevant source files
The following files were used as context for generating this wiki page:
Unpacking Artifacts refers to the systematic process of extracting, validating, and deploying firmware components onto ESP32-compatible hardware. This system ensures that encrypted Noise sessions, partition tables, and application binaries are correctly placed within the device's flash memory to enable communication with the Muse VM.
The process encompasses the initial GitHub Action build artifacts, the local flashing procedure using esptool, and the internal firmware logic that handles incoming data frames over WebSocket connections.
Firmware Component Structure
The project builds several distinct binary artifacts that must be flashed to specific offsets in the ESP32 memory. These components work together to provide a secure, OTA-capable environment.
| Artifact | Flash Offset | Description |
|---|---|---|
bootloader.bin | 0x1000 | The primary bootloader for the ESP32 chip. |
partition-table.bin | 0x10000 | Defines the layout of the flash memory, including OTA slots and NVS. |
ota_data_initial.bin | 0x17000 | Initializes the Over-the-Air (OTA) update state. |
phy_init_data.bin | 0x19000 | Contains physical layer configuration for radio (Wi-Fi/BT) hardware. |
muse-gadget.bin | 0x20000 | The main application binary containing the project logic. |
Sources: workflows/build-release.yml:55-63, README.md:83-93
Deployment and Flashing Flow
The deployment process begins with the compilation of source code into artifacts, followed by the transmission of these binaries to the device hardware.
Automated Build Pipeline
The GitHub Actions workflow automates artifact generation. It clones the SDK, applies the specific board-port overlays (such as the CYD board support), and builds the firmware using the espressif/idf:v6.0.1 container. The workflow injects the GADGET_SDK_TOKEN secret during compilation to authorize the device for Muse pairing.
Sources: workflows/build-release.yml:32-53
Flashing Procedure
You flash the unpacked artifacts onto the board using esptool. The process requires a USB data cable and auto-detects the serial port.
The diagram shows the sequence of operations performed by the flash.sh script to deploy artifacts.
Sources: README.md:79-106, board-port/esp32/AGENTS.md:278-285
Inbound Data Unpacking
Once the firmware is running, it "unpacks" inbound data artifacts received through the Noise-encrypted WebSocket session. The noise_control.cpp module manages the decryption and routing of these service frames.
Service Frame Processing
The firmware receives binary WebSocket frames and processes them through a ClientSession. If the frame is valid, the system identifies the stream ID and dispatches the payload to the appropriate handler.
- Stream 1 (Control): Handles JSON-encoded commands like
device.healthordisplay.draw_url. - Stream 2 (Tunnel): Handles raw IP packets for the home-network tunnel.
- Stream 3 (Identity): Reassembles the agent's name and metadata from JSON fragments.
Sources: board-port/esp32/main/noise_control.cpp:55-62, board-port/esp32/main/noise_control.cpp:924-970
JSON Command Dispatch
When a control frame arrives, the firmware reassembles the u32-LE length-prefixed fragments into a complete JSON object. The handle_invoke_request function parses this object to execute specific methods.
The diagram illustrates how the system unpacks encrypted network artifacts into actionable application commands.
Sources: board-port/esp32/main/noise_control.cpp:862-895, board-port/esp32/main/noise_control.cpp:1115-1160
OTA Artifact Validation
After a new artifact is unpacked and flashed via an Over-the-Air (OTA) update, the system enters a PENDING_VERIFY state. The firmware must validate the new image within 300 seconds (5 minutes) by successfully establishing a control WebSocket connection.
If the connection fails within the OTA_VERIFY_TIMEOUT_US period, the ota_verify_task triggers a rollback. This revert process ensures the device remains recoverable by switching back to the previously known-good firmware partition.
Sources: board-port/esp32/main/app.c:1346-1372, board-port/esp32/main/app.c:116-119
Summary
Unpacking artifacts involves both the physical deployment of binary images to ESP32 flash and the logical extraction of encrypted data from network streams. The system relies on fixed memory offsets for deployment and specific stream identifiers for runtime data routing, providing a secure method for device management and feature execution.