Training Setup & Prerequisites
Relevant source files
The following files were used as context for generating this wiki page:
Training in the Piper project involves preparing high-quality audio data and corresponding transcriptions to train a VITS-based neural text-to-speech model. The system relies on PyTorch Lightning for the training pipeline and requires specific data structures and configurations to handle phonemization, audio normalization, and multi-speaker mapping.
The setup ensures that raw text and audio are converted into cached tensors (phoneme IDs, mel-spectrograms, and normalized audio) before the training process begins, optimizing the data flow for high-performance training sessions.
1. System Requirements and Dependencies
The Piper training module requires a specific set of Python packages. These are defined as an extra requirement in the project's setup configuration.
Software Prerequisites
- Python: Version >= 3.9 is required.
- Core Libraries:
torch(v2.x),lightning(v2.x), andonnx. - Audio Processing:
librosaandpysilero-vadfor voice activity detection. - Configuration:
jsonargparsefor handling complex training signatures.
Sources: setup.py:53-65, CHANGELOG.md:37-40
| Dependency | Purpose |
|---|---|
torch | Primary deep learning framework |
lightning | High-level training loop and data management |
pysilero-vad | Used for automatic silence trimming in audio |
librosa | Loading and resampling audio files |
cython | Optimizing performance-critical paths |
Sources: setup.py:54-64
2. Dataset Preparation
Piper utilizes a VitsDataModule (built on PyTorch Lightning) to manage the transition from raw files to training-ready tensors.
Input Data Format
The system expects a CSV metadata file where each row maps an audio file to its transcription.
- Single Speaker:
wav_filename|text - Multi-Speaker:
wav_filename|speaker_name|text - Pre-generated IDs:
wav_filename|text|phoneme_ids(or with speaker ID)
Sources: src/piper/train/vits/dataset.py:145-163, CHANGELOG.md:14-16
Data Processing Pipeline
The following diagram illustrates how raw input data is processed into cached training features:
The pipeline converts all inputs into .pt (PyTorch) files within a cache_dir to avoid redundant processing during training epochs.
Sources: src/piper/train/vits/dataset.py:170-250
3. Configuration & Model Hyperparameters
Training is governed by three primary configuration classes that define the audio characteristics, model architecture, and optimization strategy.
Audio Configurations
Audio is processed into Mel-spectrograms using the following default parameters:
- Sample Rate: 22050 Hz (default)
- Filter Length: 1024
- Hop Length: 256
- Win Length: 1024
Sources: src/piper/train/vits/config.py:10-18
Model Architecture Selection
Piper supports "low quality" and "high quality" presets for the VITS generator, primarily adjusting the residual blocks and upsampling rates.
| Feature | Low Quality | High Quality |
|---|---|---|
| Resblock Type | 2 | 1 |
| Upsample Rates | (8, 8, 4) | (8, 8, 2, 2) |
| Initial Channels | 256 | 512 |
| Upsample Kernels | (16, 16, 8) | (16, 16, 4, 4) |
Sources: src/piper/train/vits/config.py:32-58
4. Training Logistics
The VitsDataModule handles the splitting of data and the creation of DataLoaders.
Data Splitting and Collation
- Validation Split: Default is 10% of the dataset.
- Test Set: A fixed number of examples (default 5) are reserved for testing.
- Collation: The
UtteranceCollateclass handles dynamic padding, ensuring all sequences in a batch match the length of the longest sequence. Audio segments are padded to a minimum of 8192 samples.
Sources: src/piper/train/vits/dataset.py:315-325, src/piper/train/vits/dataset.py:375-385
Hardware and Optimization
Training supports:
- CUDA: Optional GPU acceleration via
torchandlightning. - Mixed Precision:
fp16_runtoggle inTrainingConfig. - Multi-Speaker: Mapping speaker names to integer IDs via a
speaker_id_mapstored in the model configuration.
Sources: src/piper/train/vits/config.py:117, src/piper/train/vits/dataset.py:140-155
5. Model Export Preparation
Once training is complete, the checkpoint (.ckpt) must be exported to ONNX format for inference.
Export Requirements
The export script export_onnx.py prepares the model by:
- Loading the
VitsModelfrom a checkpoint. - Removing weight normalization from the decoder.
- Setting the model to evaluation mode.
- Defining dynamic axes for
batch_size,phonemes, andtimeto allow variable-length input during inference.
Sources: src/piper/train/export_onnx.py:44-70
Sources: src/piper/train/export_onnx.py:50-85
Training setup for Piper is a structured process that prioritizes data caching and configuration-driven architecture. By ensuring all prerequisites—from software dependencies to cached tensors—are met, the system provides a robust environment for developing high-quality, local neural text-to-speech voices.