Customizing the UI Layer
Relevant source files
The following files were used as context for generating this wiki page:
The UI Layer provides visual and auditory feedback regarding the device's connection status, pairing state, and Muse interactions. It scales across different hardware profiles, ranging from simple RGB status LEDs to high-resolution LCD panels featuring pixel-art animations and text captions.
Customizing the UI involves modifying how the device represents internal states—such as Wi-Fi provisioning, Noise session establishment, and command execution—through various hardware backends.
Sources: board-port/esp32/README.md, board-port/esp32/main/led_status.c:266-282
Hardware Backends
The system supports multiple UI backends selected via Kconfig. Each backend implements a common status interface but renders information differently based on hardware capabilities.
Backend Types
- LED PWM RGB: Standard RGB LED driven by three PWM channels.
- Addressable RGB (WS2812): Single or multiple LEDs (e.g., LED rings) driven via the RMT peripheral.
- SPI/RGB LCD Panels: Full-color screens (ST7789, ILI9341, ST7701) displaying animations and text.
- E-Paper: Low-power black-and-white or multi-color displays for persistent status.
- Muse UI: Full on-screen UI featuring an animated avatar, settings menu, and captions.
Sources: board-port/esp32/main/led_status.c:38-84, board-port/esp32/main/CMakeLists.txt:58-62, board-port/esp32/README.md
Device Status Mapping
The UI layer translates internal led_state_t values into specific visual patterns.
| State | Visual Feedback | Meaning |
|---|---|---|
LED_STATE_BLE_ADVERTISING | Orange, breathing | BLE advertising; waiting for phone setup |
LED_STATE_PAIRING_CONFIRM_REQUIRED | Blue, breathing | Needs physical button press to confirm pairing |
LED_STATE_WS_CONNECTED | Green (Steady or Dot) | Connected to Muse; tunnel/control session live |
LED_STATE_WS_DISCONNECTED | Yellow, blinking | Connection lost; attempting to reconnect |
LED_STATE_ERROR | Red, blinking | Hardware or protocol error encountered |
Sources: board-port/esp32/main/led_status.c:858-912, board-port/esp32/README.md
UI Data Flow
The led_task serves as the primary actor for UI updates, receiving state changes from the application logic (app.c) or network control (noise_control.cpp).
The led_task monitors global state variables and periodically refreshes the hardware output to reflect breathing effects, blinks, or frame-by-frame animations.
Sources: board-port/esp32/main/led_status.c:854-915, board-port/esp32/main/app.c:1347-1355
LCD Customization
On boards with a display, the UI layer manages edge bars, a status dot, and a title text area.
Animation and Layout
The anim_task renders a pixel-art animation (e.g., happy_anim) in the center of the screen. Status colors are applied to "bars" at the top and bottom of the vertical resolution. Once connected, a small green dot replaces the status bars to indicate an active session.
Sources: board-port/esp32/main/led_status.c:284-306, board-port/esp32/main/led_status.c:646-653
Text Rendering
Captions and titles use a built-in pixel font. The system dynamically scales the font size based on string length to ensure the text fits within the screen width.
// Example of title scaling in led_status.c
int scale = LCD_TITLE_MAX_SCALE;
while (scale > 2 && n * adv * scale - scale > LCD_H_RES) scale--;
Sources: board-port/esp32/main/led_status.c:595-600
Remote Display Commands
Muse can remotely control the UI via WebSocket commands. The application handles these by pausing local animations to draw external content.
display.draw_url: Fetches a JPEG or raw RGB565 image and draws it at a specified row.display.text: Sets the title text shown on the display.display.show_animation: Clears external images and returns the screen to its default animation state.
Sources: board-port/esp32/main/app.c:977-1013, board-port/esp32/main/noise_control.cpp:525-580
Voice UI Integration
Devices with audio support include a "Voice UI" mode. This mode overrides standard status indications to provide real-time feedback during voice interactions.
The LED ring represents audio input levels by growing a meter from the top down both sides. Distinct "comet" animations (rotating heads with fading tails) represent transcription and processing states.
Sources: board-port/esp32/main/led_status.c:805-846, board-port/esp32/main/noise_control.cpp:602-615
Adding a Custom UI Backend
To add support for a new hardware UI, you must:
- Define the new backend in
Kconfig.projbuild. - Add required drivers to
main/CMakeLists.txt. - Implement hardware-specific initialization in
led_hw_init(). - Implement
led_hw_set_color(rgb_t c)to map the RGB status color to your hardware (e.g., setting PWM duty cycles or RMT pixels).
Sources: board-port/esp32/main/led_status.c:67-84, board-port/esp32/main/CMakeLists.txt:38-55, board-port/esp32/AGENTS.md
The UI layer enables a consistent user experience across diverse ESP32 hardware, ensuring that pairing, connection, and AI interaction states remain clear regardless of the specific display or LED configuration used.