Monotonic Alignment Build
Relevant source files
The following files were used as context for generating this wiki page:
The Monotonic Alignment Build refers to the specialized compilation and integration process for the monotonic alignment search module used within the Piper text-to-speech engine. This system is critical for mapping phonemes to specific durations in the generated audio waveform, ensuring that the speech timing is accurate and matches the linguistic input. In the Piper project, this functionality is primarily utilized during the training phase of VITS models and for experimental alignment support in inference.
The build process involves compiling Cython-based C extensions to achieve the performance necessary for high-speed alignment calculations. These alignments allow the engine to determine how many audio samples correspond to each phoneme id, providing granular timing data.
Sources: build_monotonic_align.sh:1-12, CHANGELOG.md:27-27, src/piper/voice.py:270-320
Build Architecture and Compilation
The monotonic alignment module is implemented using Cython to interface with optimized C code. This is necessary because the alignment search is a computationally intensive task that would be too slow if implemented in pure Python.
Compilation Workflow
The build script automates the environment setup and compilation. It targets a specific directory within the Piper training source tree and uses cythonize to generate C code from .pyx source files, which is then compiled into a shared object (.so) file.
The workflow ensures that the performance-critical "core" logic is compiled specifically for the host architecture.
Sources: build_monotonic_align.sh:1-12
Alignment in Data Processing
During the training phase, alignments are managed by the data module. The system expects alignment data to be present in a specific directory structure for processing during model training.
Dataset Configuration
The VitsDataModule handles the paths and requirements for alignments. If an explicit alignments directory is not provided, the system defaults to an alignments sub-directory relative to the input CSV file.
| Component | Description | Default / Source |
|---|---|---|
alignments_dir | Directory containing pre-calculated alignment files. | csv_path.parent / "alignments" |
hop_length | Number of samples between successive frames, used for alignment scaling. | 256 |
segment_size | Minimum audio segment size (e.g., 8192) for processing. | 8192 |
Sources: src/piper/train/vits/dataset.py:84-88, src/piper/train/vits/config.py:10-14
Model Patching for Alignment Output
Piper provides a utility to "patch" existing ONNX voice models to enable phoneme width alignment output. This is useful for models that were originally exported without alignment tensors.
Tensor Detection and Modification
The patching script looks for specific operations within the ONNX graph—specifically "Ceil" nodes—which are often associated with the duration or alignment logic of the model. By marking these internal tensors as model outputs, the engine can extract timing information during inference.
Alignment Logic during Synthesis
When synthesizing audio, the PiperVoice class can optionally return phoneme-to-audio alignments if the underlying model supports it.
The PhonemeAlignment Structure
Alignments are represented as a sequence of mappings between phonemes, their internal IDs, and the resulting number of audio samples.
@dataclass
class PhonemeAlignment:
phoneme: str
phoneme_ids: Sequence[int]
num_samples: int
Sources: src/piper/voice.py:28-32
Alignment Extraction Flow
During synthesis, the engine performs the following steps to generate alignment data:
- Inference: The ONNX model is run with
include_alignments=True. - Sample Calculation: The raw alignment output from the model is multiplied by the
hop_length(typically 256) to convert frame-based durations into sample counts. - Mapping: The engine iterates through the phonemes (including special tokens like
BOSandEOS) and groups the sample counts from the model's output to match the linguistic input.
Sources: src/piper/voice.py:270-320, src/piper/voice.py:461-468
Summary
The Monotonic Alignment Build system bridges the gap between high-level linguistic input and low-level audio sample timing. By utilizing Cython for high-performance builds, providing dataset integration for training, and offering model patching utilities for inference, Piper ensures that timing information remains accessible throughout the text-to-speech pipeline. This system allows developers to obtain precise timestamps for when each phoneme is spoken in the generated audio.
Sources: build_monotonic_align.sh, src/piper/voice.py, CHANGELOG.md