ESP32 Firmware Overview
Relevant source files
The following files were used as context for generating this wiki page:
The ESP32 firmware is an open-source SDK that connects ESP32-compatible hardware to the Muse platform via home Wi-Fi. It enables devices to show images, process audio input/output, and interact with the home network through an encrypted tunnel. The firmware supports a wide range of hardware configurations, from basic status LEDs to complex devices featuring animated avatars and touch-screen user interfaces.
The SDK is built using Espressif's ESP-IDF version 6.0.1. It provides a modular architecture where device-specific logic is isolated into board files and overlays, while core functionality like Wi-Fi management, Bluetooth pairing, and the Muse UI remains shared across the platform.
Sources: esp32/README.md:16-30, esp32/README.md:88-95
Firmware Architecture and Types
The firmware categorizes supported devices into four distinct "kinds" based on their hardware capabilities and the level of UI feedback they provide.
| Kind | Status Feedback | Implementation Requirement |
|---|---|---|
| Light | Status LED | sdkconfig overlay |
| Status Screen | Edge bars and animations | Overlay + display backend in main/ |
| E-paper | Text-based status screen | Overlay + specific led_status.h |
| UI | LVGL avatar, voice, and settings | Overlay + muse_board_t implementation |
Sources: esp32/devices/AGENTS.md:160-165
Core Components Flow
The firmware initializes hardware through a standardized board interface, then starts the Muse core services.
The diagram shows the sequential initialization process from the main application entry point to the active UI state.
Sources: esp32/components/muse/boards/board_vn_s3_183.c:120-220, esp32/README.md:148-160
Hardware Abstraction Layer (muse_board_t)
Devices with the full UI implement the muse_board_t structure. This structure defines how the firmware interacts with specific hardware components like displays, buttons, and power management units (PMU).
Key Structure Fields
- init: Executes first to configure power latches, I2C buses, and the PMU.
- display_start: Initializes the panel, LVGL, and input devices.
- audio_init: Configures the
esp_codec_devfor speaker and microphone operations. - poll_buttons: Returns button edges (e.g.,
MUSE_BTN_TALK_PRESS) to the core system. - read_power: Measures battery percentage and charging status.
Sources: esp32/devices/AGENTS.md:200-245, esp32/components/muse/boards/board_aipi.c:265-288
Hardware Configuration Example
The board implementation for the VN ESP32-S3 1.83-inch board demonstrates the typical initialization of SPI displays and I2C codecs:
static const muse_board_t s_board = {
.name = "VN ESP32-S3 1.83-inch NV3023",
.width = 284,
.height = 240,
.talk_button = "boot",
.aux_button = "Vol+",
.init = init,
.display_start = display_start,
.audio_init = audio_init,
.poll_buttons = poll_buttons,
.read_power = read_power,
};
Sources: esp32/components/muse/boards/board_vn_s3_183.c:305-325
User Interface and Settings
The UI is built using the LVGL library. It features an animated avatar and a multi-page settings menu for device configuration. The settings system is designed to be memory-efficient, building sub-pages only when they are opened and deleting them when the user navigates back to save approximately 50 KB of RAM.
Settings Navigation Structure
The diagram illustrates the hierarchical navigation within the Muse Settings UI.
Sources: esp32/components/muse/muse_settings_ui.c:56-78, esp32/components/muse/muse_settings_ui.c:286-302
Input and Text Entry
The firmware includes a specialized text entry system that adapts to screen size. On screens larger than 2 inches, it provides a full lv_keyboard. For smaller screens, it utilizes the muse_keypad component to ensure buttons remain large enough for interaction.
Sources: esp32/components/muse/muse_settings_ui.c:380-410
Power and Battery Management
The firmware supports various battery sensing methods, ranging from basic ADC voltage dividers to dedicated Fuel Gauges like the CW2017 or PMUs like the AXP2101.
- ADC Divider: Used on boards like the AIPI Lite, where raw 12-bit counts are calibrated to battery percentages.
- PMU Integration: Boards like the Waveshare C6-206 utilize the AXP2101 for power off and reading battery status via I2C.
- Deep Sleep: The
power_offfunction typically puts the panel to sleep and enters ESP32 deep sleep, configured to wake on specific button GPIO edges.
Sources: esp32/components/muse/boards/board_aipi.c:228-258, esp32/components/muse/boards/board_vn_s3_183.c:285-300, esp32/devices/AGENTS.md:240-245
Build and Deployment
The firmware uses a tiered configuration system:
- sdkconfig.defaults: Global default settings for the SDK.
- sdkconfig.muse: Core UI and service settings.
- sdkconfig.muse-[board]: Board-specific overrides (e.g., PSRAM mode, flash size).
Build Procedure
- Select the board target using the
tools/board.shortools/muse/board.shhelpers. - Define the
CONFIG_GADGET_SDK_TOKENfor Muse platform authentication. - Compile using
idf.py build. - Flash via the native USB Serial/JTAG port or an external bridge (CH340/CH9102).
Sources: esp32/README.md:120-140, esp32/devices/AGENTS.md:175-188
Summary
The ESP32 firmware provides a robust, extensible framework for building Muse-enabled IoT devices. By abstracting hardware differences into a unified board structure and optimizing UI components for low-memory environments, the SDK allows developers to port the firmware to diverse ESP32 hardware while maintaining consistent behavior and connectivity to the Muse ecosystem.
Sources: esp32/README.md:200-220