Voice Call Mode & Audio
Relevant source files
The following files were used as context for generating this wiki page:
Voice Call Mode provides a hands-free interface for interacting with AI models. It integrates Voice Activity Detection (VAD), Text-to-Speech (TTS), and specific audio routing to simulate a real-time telephone conversation experience. The system handles background noise, interruptions (barge-in), and ensures audio continues playing correctly when the application is backgrounded.
Sources: README.md, assets/release_notes/en.json
Voice Activity Detection (VAD)
The VAD system identifies human speech within an audio stream to trigger transcription or model processing. It uses Silero VAD models (versions v4 and v5) through ONNX Runtime.
VAD Architecture and Logic
The VAD module processes PCM16 audio data at a fixed 16kHz sample rate. It transitions through internal states based on speech probability thresholds.
- Positive Threshold: Probability above which speech is considered started (default 0.5).
- Negative Threshold: Probability below which speech is considered ended (default 0.35).
- Minimum Speech Frames: Required duration to prevent "misfires" from short noises.
Sources: third_party/vad/UPSTREAM_README.md:166-180, third_party/vad/CHANGELOG.md
The diagram shows the logic flow from raw audio input to high-level speech events.
Sources: third_party/vad/UPSTREAM_README.md:280-310
Key VAD Events
| Event | Description |
|---|---|
onSpeechStart | Triggered when the initial probability exceeds the positive threshold. |
onRealSpeechStart | Triggered when speech is confirmed by meeting the minimum frame duration. |
onSpeechEnd | Triggered when probability stays below the negative threshold for the redemption period. |
onVADMisfire | Triggered if speech starts but ends before reaching the minimum frame count. |
onFrameProcessed | Provides real-time speech probabilities and raw audio for every frame. |
Sources: third_party/vad/UPSTREAM_README.md:330-355
Text-to-Speech (TTS) Bridges
Conduit utilizes native TTS engines for Android and iOS to provide voice responses. These bridges handle audio session management and synchronization with the UI.
Android TTS Implementation
The NativeTtsBridge manages the Android TextToSpeech engine. It implements a SilentCallAudioKeepAlive to prevent Android 12+ from dropping the audio route during pauses between sentences.
- Audio Routing: Uses
USAGE_VOICE_COMMUNICATIONandCONTENT_TYPE_SPEECHfor calls. - Keep-Alive: Plays a looped silent
AudioTrackto maintain the app's ownership of the communication audio mode. - Grace Period: Maintains the keep-alive for 5,000ms after an utterance ends to bridge gaps in streamed responses.
Sources: android/app/src/main/kotlin/app/cogwheel/conduit/NativeTtsBridge.kt:280-350, android/app/src/main/kotlin/app/cogwheel/conduit/NativeTtsBridge.kt:450-480
iOS TTS Implementation
The NativeIosTtsBridge uses AVSpeechSynthesizer. It supports personal and novelty voices introduced in iOS 17.
- Activation: Activates
AVAudioSessionwith.notifyOthersOnDeactivationwhenvoiceCallis requested. - Start Acknowledgement: Implements a 2-second timeout for starting utterances to ensure the bridge remains responsive if the system TTS engine hangs.
- Progress Tracking: Emits progress events including character ranges to the Flutter layer for UI highlighting.
Sources: ios/Runner/NativeIosTtsBridge.swift:105-140, ios/Runner/NativeIosTtsBridge.swift:300-325
Audio Routing and Lifecycle
Voice calls require specific platform configurations to support speakerphone and hardware integration.
Call Mode Transitions
When a voice call starts, the app modifies audio settings to prioritize the communication stream.
The sequence shows how the native bridges manage system audio focus and communication modes during a call.
Sources: android/app/src/main/kotlin/app/cogwheel/conduit/NativeTtsBridge.kt:360-400, ios/Runner/NativeIosTtsBridge.swift:145-160
Configuration Parameters
The system allows fine-tuning of audio behavior through method call arguments.
| Parameter | Type | Default | Description |
|---|---|---|---|
rate | Float | 0.5 | Speech speed; ranges vary by platform (e.g., 0.1 to 3.0). |
pitch | Float | 1.0 | Voice pitch; typically 0.5 to 2.0. |
volume | Float | 1.0 | Output volume; 0.0 to 1.0. |
voiceCall | Boolean | false | If true, applies communication routing and keep-alive logic. |
Sources: android/app/src/main/kotlin/app/cogwheel/conduit/NativeTtsBridge.kt:130-150, ios/Runner/NativeIosTtsBridge.swift:150-170
Implementation Details
Silent Call Audio Keep-Alive (Android)
Android 12 (API 31) and newer require an active audio track for the app to remain the owner of MODE_IN_COMMUNICATION. The SilentCallAudioKeepAlive class creates a static AudioTrack containing 0.5 seconds of silence (PCM16, Mono, 16kHz) and loops it indefinitely. This prevents the system from reverting to MODE_NORMAL while the TTS engine (which runs in a separate process) is speaking.
Sources: android/app/src/main/kotlin/app/cogwheel/conduit/NativeTtsBridge.kt:492-540
TTS File Synthesis
Both platforms support synthesizing text to a local file instead of immediate playback.
- Android: Uses
tts.synthesizeToFileto create a.wavfile in the cache directory. - iOS: Uses
synthesizer.writeto create a.caffile in the temporary directory. - Timeout: Both implementations include a 60-120 second timeout to prevent stalled synthesis tasks from leaking resources.
Sources: android/app/src/main/kotlin/app/cogwheel/conduit/NativeTtsBridge.kt:70-100, ios/Runner/NativeIosTtsBridge.swift:330-360
Voice Call Mode ensures a continuous conversational flow by managing VAD for input and native TTS bridges for output, with specialized handling for platform-specific audio routing requirements.