Energy-based Audio Chunking
Relevant source files
The following files were used as context for generating this wiki page:
Energy-based audio chunking is a critical preprocessing mechanism within the mlx-qwen3-asr project, designed to handle long-form audio by splitting it into manageable segments for the transformer-based encoder-decoder model. This process ensures that audio inputs do not exceed the model's architectural limits or the memory constraints of Apple Silicon hardware.
The system utilizes signal energy levels to identify optimal "silence" or low-energy regions for splitting. By placing boundaries at these points, the system minimizes the risk of cutting off words or phonemes, which significantly improves the transcription quality of long-form media. This mechanism is integrated into both the offline transcription pipeline and the experimental streaming decode features.
Sources: mlx_qwen3_asr/transcribe.py:202-210, mlx_qwen3_asr/chunking.py, CLAUDE.md:144-145
Architecture and Logic
The chunking logic is primarily housed in mlx_qwen3_asr/chunking.py and is invoked during the transcription pipeline in mlx_qwen3_asr/transcribe.py. When an audio file exceeds a specific duration (defaulting to 1200 seconds or 20 minutes in many contexts), the system triggers a recursive or iterative split based on energy analysis.
Chunking Workflow
The following flowchart illustrates the decision-making process when handling an audio input:
The logic ensures that chunks are non-overlapping and contiguous, preserving the total sample count of the original audio.
Sources: mlx_qwen3_asr/chunking.py, tests/test_chunking.py:65-71, mlx_qwen3_asr/transcribe.py:461-463
Key Components
Core Functions
| Function | File | Description |
|---|---|---|
split_audio_into_chunks | mlx_qwen3_asr/chunking.py | The main entry point that iterates through audio and yields (chunk, offset) tuples. |
_find_split_point | mlx_qwen3_asr/chunking.py | Internal utility that calculates energy across a search range to find the quietest point. |
feed_audio | mlx_qwen3_asr/streaming.py | In streaming mode, uses energy endpointing to prefer silence boundaries for partial decodes. |
Sources: mlx_qwen3_asr/chunking.py:1-20, mlx_qwen3_asr/streaming.py:149-165, tests/test_chunking.py:108-112
Energy Detection Logic
The _find_split_point function operates by searching for a low-energy region, typically within a window centered around the midpoint of a potential chunk. By default, it looks within a range (e.g., 20% to 80% of the segment) to avoid splitting near the very beginning or end of a clip.
def _find_split_point(audio, sr):
# Logic to identify the index with lowest RMS or absolute energy
# over a sliding window.
# Returns the index of the optimal split.
Sources: mlx_qwen3_asr/chunking.py, tests/test_chunking.py:126-140
Integration with Transcription Pipelines
Offline Transcription
In the transcribe function, audio is first converted to a float32 numpy waveform at 16kHz before being passed to the chunker. The resulting chunks are processed individually:
- Mel Spectrogram Generation: Each chunk is converted to 128 bins.
- Encoding: Audio features are extracted per chunk.
- Decoding: The text is generated autoregressively.
- Merging: Chunk texts are joined, using CJK-aware joiners (no spaces for Chinese/Japanese/Korean) or standard spaces for other languages.
Sources: mlx_qwen3_asr/transcribe.py:461-530, mlx_qwen3_asr/transcribe.py:79-84
Streaming Decode
For live or streaming audio, the feed_audio function supports an energy endpointing mode. This mode uses a lookback window to detect silence before finalizing a chunk for decoding, reducing latency while maintaining stability.
Configuration and Parameters
The behavior of the energy-based chunking can be influenced by several CLI and API parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
max_chunk_sec | float | 1200.0 | Maximum duration for a single processing chunk in offline mode. |
endpointing_mode | string | "fixed" | In streaming, can be set to "energy" to use silence detection. |
endpoint_min_chunk_sec | float | 0.2 | Minimum audio required before an energy-based split is allowed. |
silence_sec | float | 0.2-0.25 | Used in manifest builders to insert deterministic gaps between clips. |
Sources: mlx_qwen3_asr/cli.py:446-460, tests/test_streaming.py:115-130, scripts/build_diarization_manifest.py:175-180
Summary
Energy-based Audio Chunking is the foundational method for processing long audio files in the mlx-qwen3-asr ecosystem. By locating low-energy regions for splitting, the system ensures high transcription accuracy and efficient memory usage on Apple Silicon. It provides a robust bridge between the raw audio signal and the transformer architecture, supporting both high-throughput offline batching and low-latency streaming applications.
Sources: CLAUDE.md:46-52, mlx_qwen3_asr/transcribe.py:586-590