Dataset Configuration
Relevant source files
The following files were used as context for generating this wiki page:
Dataset configuration in Piper defines the environment and parameters required to process raw text and audio into a format suitable for training and inference with the VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) architecture. This system manages phoneme mappings, audio preprocessing (such as silence trimming and Mel-spectrogram generation), and speaker identification for multi-speaker models.
The configuration bridge is primarily managed through VitsDataModule, which orchestrates data preparation, caching, and loading via PyTorch Lightning. It ensures consistency between the training pipeline and the final exported ONNX models by generating a standard config.json file that includes audio parameters and phoneme-to-ID mappings.
Sources: src/piper/train/vits/dataset.py:46-118, src/piper/train/vits/config.py:73-95
Core Data Structures
The system relies on several key dataclasses and enums to maintain configuration state throughout the pipeline.
Dataset and Phoneme Types
Piper supports different types of input datasets and phoneme processing strategies.
| Type | Enumeration Value | Description |
|---|---|---|
| Dataset Type | text | Standard dataset where metadata contains text for phonemization. |
| Dataset Type | phoneme_ids | Dataset using pre-generated phoneme IDs. |
| Phoneme Type | espeak | Uses espeak-ng for phonemization (default). |
| Phoneme Type | text | Uses raw text/codepoints as phonemes. |
| Phoneme Type | pinyin | Uses Chinese g2pW for Pinyin-based phonemization. |
Sources: src/piper/train/vits/dataset.py:40-43, src/piper/config.py:14-19
Model and Training Configuration
Configuration is split between architectural parameters and training hyperparameters.
@dataclass
class ModelConfig:
num_symbols: int
n_speakers: int
audio: ModelAudioConfig
inter_channels: int = 192
hidden_channels: int = 192
filter_channels: int = 768
n_heads: int = 2
n_layers: int = 6
# ...
Sources: src/piper/train/vits/config.py:73-89
Data Preparation Flow
The VitsDataModule.prepare_data method executes the primary workflow for transforming a raw CSV metadata file and audio directory into a cached training dataset.
The process involves calculating a cache_id for each utterance based on the row number, text, and speaker ID to avoid redundant processing.
Sources: src/piper/train/vits/dataset.py:120-255
Audio Preprocessing Details
Audio is processed into normalized tensors and Mel-spectrograms. If trim_silence is enabled, the system uses the Silero VAD (Voice Activity Detector) at a 16kHz sample rate to identify speech boundaries, then applies user-defined buffers (keep_seconds_before_silence and after) to the original audio.
| Parameter | Default | Description |
|---|---|---|
filter_length | 1024 | Window size for FFT. |
hop_length | 256 | Number of audio samples between adjacent STFT columns. |
win_length | 1024 | Window size for STFT. |
segment_size | 8192 | Size of audio segments used during training. |
Sources: src/piper/train/vits/dataset.py:56-65, src/piper/train/vits/config.py:10-14
Multi-Speaker Configuration
For datasets containing multiple speakers, Piper implements a mapping system to translate speaker names from metadata into integer IDs.
- Metadata Format: Multi-speaker CSVs are expected to follow the format
wav|speaker|text. - ID Mapping: The
VitsDataModulescans the CSV to build aspeaker_id_map. - Conditioning: If
num_speakers > 1, the model automatically setsgin_channelsto 512 to accommodate global speaker embeddings.
Dataset and Batching Logic
Processed data is served via VitsDataset and UtteranceCollate. The collation process handles padding for variable-length sequences of phonemes, spectrograms, and audio.
Batch Structure
A Batch consists of tensors padded to the maximum length within that specific batch:
- phoneme_ids: LongTensor (batch, max_phonemes)
- spectrograms: FloatTensor (batch, mels, max_spec_length)
- audios: FloatTensor (batch, 1, max_audio_length)
- speaker_ids: Optional LongTensor (batch)
The UtteranceCollate class ensures that audio tensors are at least segment_size (default 8192) in length to satisfy model constraints.
Sources: src/piper/train/vits/dataset.py:326-340, src/piper/train/vits/dataset.py:378-400
Export and Inference Configuration
When a model is exported to ONNX via export_onnx.py, the dataset configuration (specifically num_symbols and num_speakers) dictates the input shapes and dynamic axes of the exported model.
| Input Name | Shape | Description |
|---|---|---|
input | [batch, phonemes] | Sequence of phoneme IDs. |
input_lengths | [batch] | Actual length of each input sequence. |
scales | [3] | Noise scale, length scale, and noise-w scale. |
sid | [batch] | Speaker ID (only if multi-speaker). |
Sources: src/piper/train/export_onnx.py:99-105, src/piper/voice.py:465-485
Conclusion
The Dataset Configuration system provides a robust framework for managing the transition from raw linguistic and acoustic data to structured tensors. By centralizing phoneme mapping, audio normalization, and multi-speaker logic within the VitsDataModule and associated config classes, Piper ensures that training remains reproducible and consistent with the requirements of the VITS architecture and ONNX runtime inference.