Audio/Video Device Skills
Relevant source files
The following files were used as context for generating this wiki page:
Audio/Video Device Skills enable Muse Gadgets to read status and control playback for media devices through Muse Home Link. These community-sourced integrations interact with speakers, streaming players, and smart TVs via local network protocols.
Users provide a link to the skills repository or specific skill files in the Muse chat to enable these capabilities. The system prioritizes local TCP/UDP communication and adheres to safety rules defined in the Home Link documentation.
Sources: skills/README.md:1-12, skills/CATALOG.md:1-24, esp32/README.md:1-15
Core A/V Protocols
The SDK organizes A/V functionality around standard communication protocols. Each skill targets specific hardware but often shares underlying connection logic for Cast, SSAP, or Remote v2 interfaces.
Google Cast Integration
The Google Cast skill serves as a shared procedure for devices that advertise a _googlecast._tcp service. It handles media playback, volume control, and receiver status for individual speakers and existing speaker groups.
This diagram shows the operational flow for interacting with a Google Cast receiver from discovery to playback verification.
Sources: skills/gadget-google-cast/SKILL.md:7-40, skills/CATALOG.md:8-9
Smart TV Control (LG webOS)
LG webOS TV skills use the SSAP WebSocket service for local control. This interface manages inputs, installed applications, and remote button functions.
- Pairing: The skill initiates a local SSAP WebSocket pairing flow that requires on-screen user approval.
- Security: The system stores and reuses the returned client key for subsequent sessions.
- Routing: If a TV advertises a Cast receiver independently, the system routes media tasks to the shared Google Cast skill.
Sources: skills/gadget-lg-webos-tvs/SKILL.md:10-38
Remote Interface Control
Devices like the Google TV Streamer combine multiple protocols for comprehensive control. They use the Android TV Remote v2 protocol for navigation and app launching, while relying on Cast for media transport.
The sequence diagram illustrates the authenticated pairing and control flow for Android TV Remote v2 devices.
Sources: skills/gadget-google-tv-streamer/SKILL.md:11-30
Implementation Details
Skills define specific Python client libraries and operational boundaries to ensure compatibility.
Client Libraries and Services
| Device Category | Primary Protocol | Client Library / Interface |
|---|---|---|
| Google Cast | Cast v2 | PyChromecast |
| LG webOS TVs | SSAP (WebSocket) | bscpylgtv |
| Google TV Streamer | Remote v2 / Cast | androidtvremote2 |
| General A/V | MDNS / TCP | Home Link shared network access |
Sources: skills/gadget-google-cast/SKILL.md:38-42, skills/gadget-lg-webos-tvs/SKILL.md:46-47, skills/gadget-google-tv-streamer/SKILL.md:33-34
Supported Media Operations
The shared Cast procedure standardizes operations across different hardware models:
- Status Reading: Reports volume, mute state, and the currently running application.
- Playback Control: Executes pause, resume, seek, and stop commands via the media controller session.
- Media Loading: Launches the Default Media Receiver (
CC1AD845) to load media URLs with specific MIME types. - Volume Management: Normalizes volume levels to the receiver-level documented range.
Sources: skills/gadget-google-cast/SKILL.md:43-60
System Limits and Safety
A/V skills operate under strict functional and safety constraints to protect user privacy and hardware integrity.
- Network Boundaries: Skills only communicate with devices on the local network that are already provisioned.
- Media Access: A Cast receiver must be able to fetch media URLs independently; the Muse agent does not automatically host local workspace files for casting.
- No Configuration: Skills do not perform firmware updates, network reconfigurations, or account setups.
- Model Restrictions: Capabilities like power-on behavior or microphone access are model-dependent and are not guaranteed by the protocol alone.
Sources: skills/gadget-google-cast/SKILL.md:20-30, 68-75, skills/gadget-lg-webos-tvs/SKILL.md:40-44
Summary
Audio/Video Device Skills provide a standardized way for Muse Gadgets to act as local controllers for home entertainment systems. By leveraging shared protocols like Google Cast and device-specific APIs like LG's SSAP, the SDK enables complex media orchestration while maintaining local network security and device safety.
Sources: skills/README.md:1-12, skills/CATALOG.md:1-24