User Interface & Avatar
Relevant source files
The following files were used as context for generating this wiki page:
The User Interface (UI) and Avatar system provide the visual identity and configuration environment for Muse gadgets. This system manages a procedural 64x64 pixel-art renderer for the avatar and an LVGL-based settings interface for device management.
Sources: esp32/tools/muse/AVATAR_RECIPE.md:17-21, esp32/components/muse/muse_settings_ui.c:56-61
Avatar System Architecture
The avatar is a procedural renderer that generates a 64x64 pixel grid. It reflects the device's current state through specific animations and visual behaviors.
Rendering Logic
The renderer uses a palette-based approach with at most 32 color entries. It utilizes a 4x4 Bayer ordered dithering technique for shading and soft edges. Performance is optimized for ESP32 hardware, targeting under 10 ms per frame on S3 chips and under 40 ms on C6 chips by using integer fixed-point arithmetic instead of floating-point operations in per-pixel loops.
Sources: esp32/tools/muse/avatar_prompt.md:52-58, esp32/tools/muse/avatar_prompt.md:73-81, esp32/tools/muse/avatar_prompt.md:83-88
Animation Modes
The avatar changes its appearance based on the muse_mode_t enumeration. Each mode triggers distinct visual beats and "schemes" (color palettes).
| Mode | Visual Behavior |
|---|---|
| BOOT | Pops up from a squash; sparkles appear sequentially. |
| IDLE | Gentle breathing, random blinks, and swaying limbs/wings. |
| LISTENING | Wide eyes, raised brows, and dotted rings that grow with audio level. |
| THINKING | Glancing eyes, "hmm" mouth, and three stepping thought dots. |
| SPEAKING | Mouth opens with audio level; body bobs and arms gesture. |
| ERROR | X-shaped eyes, red color scheme, and a side-to-side shake. |
| OFF | Waves goodbye as eyes close and glow fades. |
Sources: esp32/tools/muse/avatar_prompt.md:38-42, esp32/tools/muse/avatar_prompt.md:104-123
The Avatar Pipeline
The following diagram illustrates the flow from device state to the final screen output:
The system calculates the pose based on time (t), mode duration (mode_t), and audio levels (level).
Sources: esp32/tools/muse/avatar_prompt.md:43-50, esp32/tools/muse/avatar_prompt.md:83-88
Settings Interface
The Settings UI is built using the LVGL library. It is organized into a main "Home" page with several sub-pages for specific configurations. To save RAM on devices without PSRAM, sub-pages are built when opened and deleted when closed.
Sources: esp32/components/muse/muse_settings_ui.c:58-61, esp32/components/muse/muse_settings_ui.c:859-865
UI Components and Navigation
The interface uses a tile-based system. Users navigate by swiping left from the avatar to reach settings.
- Home Page: Provides quick access to Wi-Fi, Muse account (Hatch), Bluetooth, Sound, Sleep, and Battery settings.
- Sub-pages: Detailed controls for networking, brightness, and device tokens.
- Gestures: A right swipe on a sub-page returns the user to the Home page.
Sources: esp32/components/muse/muse_settings_ui.c:134-142, esp32/components/muse/muse_settings_ui.c:859-875, esp32/components/muse/muse_ui.h:35-37
Configuration Management
The UI allows users to modify device behavior through various controls:
- Network: Scan and join Wi-Fi networks or configure the Muse server host.
- Sound: Adjust speaker volume and microphone gain.
- Power: Monitor battery level, charge status, and configure auto-sleep timeouts.
Sources: esp32/components/muse/muse_settings_ui.c:486-501, esp32/components/muse/muse_settings_ui.c:647-662, esp32/components/muse/muse_settings_ui.c:750-763
Customization and Simulation
Custom Avatars
Users can replace the default avatar by providing a custom muse_pixel.c file in the components/muse/avatar/ directory. A Python tool (avatar.py) can generate this file by prompting Muse with a description or picture of the desired character.
Sources: esp32/tools/muse/AVATAR_RECIPE.md:21-25, esp32/tools/muse/AVATAR_RECIPE.md:33-40
UI Simulator
A desktop simulator provides a preview of the Muse interface in a 412x412 window. It compiles the production UI and avatar code but uses host adapters for hardware services like Wi-Fi and Bluetooth.
| Feature | Simulator Behavior |
|---|---|
| Input | Mouse acts as touch; Keyboard keys F1-F7 trigger modes. |
| Output | Renders to an SDL window; supports PPM screenshots. |
| Headless | Can run via CLI for automated scenario testing. |
Sources: esp32/simulator/README.md:17-23, esp32/simulator/README.md:79-88
Verification Flow
The project includes a host preview tool and the simulator to verify visual changes without flashing hardware.
The preview.c tool renders specific states (boot, idle, error, etc.) to PPM files to check art consistency.
Sources: esp32/tools/muse/preview.c:21-25, esp32/tools/muse/preview.c:54-65
Hardware Integration
The UI system interacts with hardware through the muse_board_t structure. This structure defines screen dimensions, rotation, and backlight controls.
| Field | Description |
|---|---|
width / height | Panel resolution as drawn by the UI. |
round | Boolean flag for round panels. |
set_brightness | Function pointer to control display brightness (0-100). |
panel_sleep | Function pointer to put the panel controller into sleep mode. |
Sources: esp32/devices/AGENTS.md:204-212, esp32/devices/AGENTS.md:218-223
This system ensures visual consistency across various ESP32 boards while providing the necessary tools for both local development and end-user customization.