Python Testing & Test Vectors
Relevant source files
The following files were used as context for generating this wiki page:
The Muse Gadget SDK employs a Python-based testing framework to validate both Linux and ESP32 firmware components. These tests ensure the reliability of protocol implementations, such as pairing and Noise handshakes, and verify the lifecycle of device endpoints and audio streaming capabilities.
Testing occurs primarily on the host machine without requiring physical hardware. The suite uses standard Python libraries like unittest and pytest, alongside custom C-harnesses that allow Python to execute and validate production C code in a controlled environment.
Sources: linux/README.md:108-112, esp32/README.md:195-200, esp32/tests/test_muse_voice_note.py:18-25
Testing Infrastructure and Methodologies
The project utilizes several distinct testing patterns to bridge the gap between Python scripts and C firmware.
C-in-Python Harnesses
For ESP32 specific logic, the test suite compiles production C files into local host binaries. These binaries are then executed by Python via the subprocess module. This allows testing of low-level logic—such as base64 encoding, WAV header generation, and state machine transitions—on the developer's computer.
Sources: esp32/tests/test_link_endpoint_lifecycle.py:31-45, esp32/tests/test_muse_voice_note.py:91-100
Host Simulation
The Linux SDK tests include in-process fakes for complex services. For example, test_link_client.py runs the Muse client against a fake Virtual Machine (VM) that performs a real Noise protocol handshake. This validates the encryption and communication layers without network dependencies.
Sources: linux/AGENTS.md:54-56, linux/tests/test_link_client.py
Automated Test Execution
Tests are invoked using standard Python test runners:
| Environment | Command | Requirements |
|---|---|---|
| Linux SDK | uv run --with pytest --with . pytest | uv, BlueZ, python3-dbus |
| ESP32 SDK | python3 -m unittest discover -s tests -p 'test_*.py' | mbedtls, pkg-config, IDF_PATH |
Sources: linux/README.md:110-112, esp32/README.md:197-206
Audio and Streaming Validation
The test suite includes dedicated modules for validating voice note capturing and streaming to the Muse backend. These tests ensure that audio data is correctly formatted into RIFF/WAVE headers and encoded into base64 chunks for Link transmission.
Voice Note Request Logic
The LinkVoiceNote and HatchVoiceNote tests verify the chunking of audio data. They ensure that headers, PCM data, and JSON metadata are correctly concatenated and that the resulting base64 stream matches CPython's native implementations.
The diagram shows the data flow from raw audio samples to formatted Link stream chunks.
Sources: esp32/tests/test_muse_voice_note.py:115-131, esp32/tests/test_muse_chat_link_note.py:117-123
Audio Format Constraints
The tests validate specific constants used in the audio pipeline:
- WAV Header: 44-byte PCM WAVE header, mono 16-bit, 16000Hz.
- Chunk Sizes: Link transmission uses standard chunk sizes, such as 1536 bytes for PCM stages and 2048 or 8192 bytes for base64 characters depending on the backend implementation.
- Modality: Requests are explicitly marked with
"output_modality": "text".
Sources: esp32/tests/test_muse_voice_note.py:53-56, esp32/tests/test_muse_chat_link_note.py:115-115, esp32/tests/test_muse_chat_link_errors.py:270-274
Protocol and Lifecycle Testing
Lifecycle tests ensure that device configurations persist correctly across reboots and that the pairing state transitions are handled securely.
Endpoint Lifecycle
The LinkEndpointLifecycleTest verifies how the firmware manages custom API URLs and Noise hosts. It simulates various boot scenarios, including orphaned keys and legacy marker migrations, to ensure that the device always connects to the correct backend services.
Sources: esp32/tests/test_link_endpoint_lifecycle.py:145-160
Sequence diagram illustrating the storage and retrieval of custom endpoints during the device boot lifecycle.
Error Handling in Chat Links
The suite includes stress tests for Link communication, validating behavior under various error conditions:
- Memory Failures: Simulates OOM (Out of Memory) conditions during JSON parsing and reallocation.
- Network Errors: Validates handling of HTTP 401 (Auth Required), 403 (Access Denied), and 404 statuses.
- Buffer Limits: Ensures that oversized incoming messages do not crash the system by skipping large payloads while maintaining stream synchronization.
Sources: esp32/tests/test_muse_chat_link_errors.py:240-265
Test Vectors
The system relies on JSON-formatted test vectors to ensure consistency across different implementations of the pairing protocol. For the Linux SDK, pairing tests specifically check against published fixtures located in tests/vectors/link_pairing_v5.json. These vectors provide expected inputs and outputs for:
- P-256 ECDH key exchange.
- HKDF-SHA256 key derivation.
- AES-256-GCM encryption/decryption.
Sources: linux/AGENTS.md:52-54
Summary
Python testing in the Muse Gadget SDK provides a comprehensive safety net for firmware development. By combining Python's high-level testing libraries with local C-compilation and predefined test vectors, the project achieves rigorous validation of security protocols, audio processing, and device lifecycle management on standard development machines.