Native Floating Pill Overlays
Relevant source files
The following files were used as context for generating this wiki page:
- apps/desktop/src-tauri/src/pill_process.rs
- apps/desktop/src-tauri/src/platform/macos/overlay.rs
- packages/rust_macos_pill/src/constants.rs
- packages/rust_gtk_pill/src/constants.rs
- apps/desktop/src/components/root/OverlaySyncSideEffects.ts
- apps/desktop/src-tauri/src/platform/linux/overlay.rs
- apps/desktop/src-tauri/src/platform/windows/overlay.rs
Native Floating Pill Overlays are platform-specific, lightweight visual indicators used in mausVoice to provide real-time feedback during dictation and AI assistant interactions. Unlike standard webview windows, these overlays are implemented in Rust to ensure high performance, low resource usage, and the ability to float above all other application windows natively across macOS, Windows, and Linux.
The system follows a "Rust is the API, TypeScript is the Brain" architecture. The TypeScript layer manages the application state (via Zustand) and synchronizes it with the native overlays, while the Rust layer handles the low-level rendering (e.g., via Cocoa on macOS or GTK on Linux) and process IPC.
Sources: llms.txt, apps/desktop/src/components/root/OverlaySyncSideEffects.ts:6-15, apps/desktop/src-tauri/src/pill_process.rs:165-180
Architecture and Process Lifecycle
The overlay system operates differently depending on the host operating system. On macOS, the pill is typically embedded and started on the main thread. On Windows and Linux, the system spawns a separate sidecar process (mausvoice-windows-pill.exe or mausvoice-gtk-pill) and communicates via standard input/output (stdin/stdout) pipes.
Process Spawning and Management
The PillProcess struct manages the lifecycle of the sidecar process, including spawning the child process with piped I/O and maintaining a thread-safe mutex for stdin access to prevent write collisions.
The system ensures that the pill is fully initialized before sending commands. The wait_for_ready function polls the child's stdout for a "ready" message with a 30-second timeout.
Sources: apps/desktop/src-tauri/src/pill_process.rs:41-80, apps/desktop/src-tauri/src/platform/macos/overlay.rs:25-35, apps/desktop/src-tauri/src/platform/linux/overlay.rs:10-20
Data Synchronization and IPC
Synchronization flows from the TypeScript React environment to the native pill processes. The OverlaySyncSideEffects component watches the Zustand store and invokes Tauri commands to update the pill's state.
State Payload Structure
The system synchronizes a comprehensive state object to the pill to handle the Assistant mode, including message history, streaming status, and tool permissions.
| Field | Type | Description |
|---|---|---|
active | boolean | Indicates if the agent/recording mode is active. |
input_mode | string | Current input method (e.g., "voice", "type"). |
compact | boolean | Whether the pill should render in a minimal state. |
messages | array | List of chat messages including tool results and errors. |
streaming | object | Details about currently streaming AI responses and tool calls. |
permissions | array | Pending tool execution permissions for the user to approve. |
Sources: apps/desktop/src/components/root/OverlaySyncSideEffects.ts:18-47, apps/desktop/src-tauri/src/pill_process.rs:149-158
Communication Flow
Communication uses JSON-formatted strings over IPC channels.
Visual Constants and Rendering
The pill's visual appearance is strictly defined by shared constants to ensure consistency across platforms and the web UI.
Waveform Engine
The recording indicator uses a sine-wave engine with specific tuning parameters to visualize audio levels.
| Constant | Value | Purpose |
|---|---|---|
LEVEL_SMOOTHING | 0.18 | Smoothing factor for incoming audio levels. |
WAVE_BASE_PHASE_STEP | 0.11 | The base speed of the wave animation. |
MAX_AMPLITUDE | 1.3 | Maximum vertical displacement of the wave. |
STROKE_WIDTH | 1.6 | Thickness of the wave lines. |
Sources: packages/rust_macos_pill/src/constants.rs:32-45, packages/rust_gtk_pill/src/constants.rs:59-72
Layout and Interaction Geometry
The pill changes size dynamically based on its state (Dictation vs. Assistant mode).
- Dictation Size: 200x86 px (Windows/Linux) or 200x86 px (macOS).
- Assistant Compact: ~424x120 px surface.
- Assistant Expanded: ~572x258 px surface.
- Long Press: A duration of 0.45s is required to arm the drag gesture, with a move threshold of 8.0 px to prevent accidental drags during clicks.
Sources: packages/rust_macos_pill/src/constants.rs:7-14, packages/rust_gtk_pill/src/constants.rs:8-12, packages/rust_macos_pill/src/constants.rs:180-185
Platform Implementation Details
macOS (Embedded)
The macOS implementation uses an embedded library (rust_macos_pill) and a monotonic sequence number PHASE_SEQ to ignore stale phase updates. It uses a sender/receiver channel to communicate between the Tauri process and the main-thread overlay logic.
Sources: apps/desktop/src-tauri/src/platform/macos/overlay.rs:11-55
Linux and Windows (Sidecar)
These platforms resolve binary paths in either the application resources (production) or the developer target directory (debug). Linux utilizes mausvoice-gtk-pill (likely using GTK Layer Shell), while Windows uses mausvoice-windows-pill.exe.
Sources: apps/desktop/src-tauri/src/platform/linux/overlay.rs:10-45, apps/desktop/src-tauri/src/platform/windows/overlay.rs:10-45
The Native Floating Pill Overlay system provides the low-latency visual feedback loop necessary for a voice-first interface, offloading rendering from the main webview to specialized native processes.