Voice Communication

Relevant source files

The following files were used as context for generating this wiki page:

• core/src/voice/opus.ts
• core/src/voice/voice.test.ts
• web/src/components/VoicePanel/VoicePanel.tsx
• android/app/src/main/kotlin/com/sovereign/communications/media/AudioRecorder.kt
• android/app/src/main/kotlin/com/sovereign/communications/media/AudioPlayer.kt
• core/src/protocol/message.ts
• ios/README.md
• V1_COMPLETE_TASK_LIST.md
• android/IMPLEMENTATION_SUMMARY.md
• README.md

The Voice Communication system in Sovereign Communications (SC) enables users to record, send, and play back voice messages over the decentralized mesh network. This feature is designed to be a core part of the real-time communication capabilities, alongside text and file transfers. The system prioritizes voice data in the network to ensure timely delivery.

The implementation of voice messaging is a planned feature across all platforms, with foundational work and platform-specific components in various stages of completion. Future enhancements include voice-to-text transcription.

Sources: README.md, IMPLEMENTATION_SUMMARY.md, V1_COMPLETE_TASK_LIST.md

Architecture

The voice communication feature is built upon the core messaging protocol and leverages platform-specific media handling capabilities.

Message Protocol

Voice messages are defined as a specific type within the binary message protocol. This allows the mesh network to handle them with appropriate priority.

• Message Type: VOICE
• Priority: Voice messages have a higher priority than text and file messages in the message queue, ensuring lower latency for real-time communication. The priority order is: Control > Voice > Text > File.

This diagram shows the message priority queue within the mesh network.

Encoding

The system uses different audio codecs depending on the platform, indicating a need for cross-platform compatibility handling.

Platform	Codec	Details
Core / Web	Opus	Planned for use in the core library and web application.
Android	AAC	Recorded at 64kbps with a 44.1kHz sample rate.

Sources: README.md, android/IMPLEMENTATION_SUMMARY.md

Platform Implementations

Android

The Android implementation for voice communication is the most detailed in the documentation. It includes dedicated components for recording and playback.

Sources: android/README.md, android/IMPLEMENTATION_SUMMARY.md

Components

• AudioRecorder.kt: Manages microphone access and audio recording. It encodes the audio into the AAC format.
• AudioPlayer.kt: Handles playback of received voice messages. It uses StateFlow for reactive UI updates during playback.
• Permissions: The app requires the RECORD_AUDIO permission to enable voice message recording.

Recording Flow

The following diagram illustrates the process of recording and sending a voice message on Android.

Web

The web application has planned UI components for voice messaging, but implementation is pending.

• UI Component: A VoicePanel.tsx component is designated for handling voice features.
• Encoding: The web platform is expected to use the Opus codec provided by the @sc/core library.
• Status: The feature is listed as a "Nice-to-Have" for V1.0, with UI for recording and playback pending.

Sources: V1_READINESS.md, V1_COMPLETE_TASK_LIST.md

iOS

The iOS implementation is in progress and leverages native frameworks for audio handling.

• Audio Management: The AudioSessionManager is responsible for configuring the audio session for recording and playback.
• Background Support: The app is configured for VoIP background mode, which can support persistent voice communication.
• Status: Voice message recording and playback is listed as "in progress".

Sources: ios/README.md, PROGRESS.md

Roadmap and Future Features

The voice communication system is a key part of the project's roadmap.

Feature	Status	Notes
Voice Message Recording (Android)	Implemented	Uses AAC codec.
Voice Message Playback (Android)	Implemented	Reactive state management.
Voice Message UI (Web)	Planned	VoicePanel.tsx is a placeholder.
Voice Message Recording/Playback (iOS)	In Progress	Uses AudioSessionManager.
Opus Codec Integration	Planned	For @sc/core and web.
Voice-to-Text Transcription	Planned	Slated for a future phase, potentially using Whisper.cpp.

Sources: README.md, V1_COMPLETE_TASK_LIST.md, PROGRESS.md

Summary

The voice communication feature is a significant component of the Sovereign Communications platform, with a clear architectural plan. The Android implementation is well-defined, utilizing AAC encoding and dedicated media components. The web and iOS platforms have foundational elements in place but require further development to achieve full functionality. The prioritization of VOICE packets in the mesh network underscores the importance of this feature for real-time communication within the SC ecosystem.