Chinese Phonemization

Relevant source files

The following files were used as context for generating this wiki page:

• src/piper/phonemize_chinese.py
• src/piper/voice.py
• src/piper/train/vits/dataset.py
• tests/test_chinese_phonemizer.py
• CHANGELOG.md

Chinese Phonemization in the Piper project is a specialized system designed to convert Chinese text into Pinyin-based phonemes and subsequently into unique phoneme IDs for speech synthesis. Unlike the standard espeak-ng approach used for many other languages, Chinese phonemization utilizes a dedicated ChinesePhonemizer class that leverages the g2pW (Grapheme-to-Phoneme for Mandarin) model.

Sources: CHANGELOG.md:7-10, src/piper/phonemize_chinese.py:1-15

The system handles specific Chinese linguistic challenges such as polyphone disambiguation through the g2pW converter and provides robust text normalization for numbers and temperatures using the unicode-rbnf engine. This module is integrated into the core PiperVoice synthesis pipeline and the VITS training data module.

Sources: src/piper/phonemize_chinese.py:146-160, src/piper/voice.py:155-165

Core Architecture and Data Flow

The architecture follows a multi-stage pipeline where raw Chinese text is first normalized, then converted to Pinyin syllables, split into linguistic components (initials, finals, and tones), and finally mapped to integer IDs used by the ONNX inference session.

High-Level Flow Diagram

The following diagram illustrates the transformation of text from raw input to phoneme IDs:

Text Normalization

The ChinesePhonemizer performs aggressive text normalization to ensure numbers and special symbols are spoken correctly in Mandarin. It uses regular expressions to identify patterns and the unicode-rbnf engine to convert digits into Chinese word equivalents.

Pattern Type	Regex Detail	Example Transformation
Temperature	`(?P[-−])?(?P\d+)\s*(?:°\s*C	℃)`
Percentage	`(?P...)\s*(?:%	％)`
Basic Numbers	r"-?\d+(?:\.\d+)?"	123 → 一百二十三

Sources: src/piper/phonemize_chinese.py:137-143, src/piper/phonemize_chinese.py:183-210, tests/test_chinese_phonemizer.py:130-155

Pinyin Processing Logic

Once the text is normalized, the g2pW converter generates Pinyin syllables. The system then decomposes these syllables into three distinct phonemic parts: the initial (consonant), the final (vowel/nasal), and the tone (1-5).

Sylable Decomposition Strategy

• Initials: The system checks for 24 defined initials, including "zero initials" (represented by Ø).
• Finals: 35 finals are supported, covering standard vowels, i-family, u-family, and ü-family (normalized to v for ASCII compatibility).
• Tones: Standard Mandarin tones 1 through 4, plus tone 5 for the neutral tone.

Sources: src/piper/phonemize_chinese.py:20-135, src/piper/phonemize_chinese.py:265-285

Syllable Normalization Code

The _normalize_g2pw_syllable function ensures that g2pW outputs are compatible with the Piper phoneme map:

def _normalize_g2pw_syllable(syl: str) -> str:
    m = re.match(r"^([a-züv:]+?)([1-5])$", syl)
    if not m:
        return syl
    base, tone = m.group(1), m.group(2)
    # Normalize ü-family to ASCII-friendly 'v'
    base = base.replace("u:", "v").replace("ü", "v")
    return base + tone

Sources: src/piper/phonemize_chinese.py:252-262

Phoneme ID Mapping and Padding

The final stage converts phonemic strings into integer IDs. A critical distinction in the Chinese implementation is the padding strategy.

Mapping Sequence Diagram

In Chinese phonemization, padding (typically ID 0) is inserted after a full group (Initial + Final + Tone) or after punctuation, rather than between every single phoneme. This group-based padding is defined by GROUP_END_PHONEMES, which includes tones and various punctuation marks.

Sources: src/piper/phonemize_chinese.py:115-135, tests/test_chinese_phonemizer.py:100-125

Integration with VITS Training

For model training, the VitsDataModule detects when the phoneme type is set to pinyin. It then initializes the ChinesePhonemizer and sets the phoneme_id_map to the specific Chinese PHONEME_TO_ID dictionary.

• Model Directory: The phonemizer expects the g2pW ONNX model to be located in a local/g2pW subdirectory.
• Auto-Download: The download_model utility is called upon initialization to fetch the required g2pw.onnx from HuggingFace if not present.

Sources: src/piper/train/vits/dataset.py:130-145, src/piper/phonemize_chinese.py:288-305

Dataset Preparation Workflow

1. Load CSV metadata containing Chinese text.
2. Initialize ChinesePhonemizer.
3. Normalize and phonemize text using g2pW.
4. Convert result to ID sequence using chinese_phonemes_to_ids.
5. Save phoneme IDs as PyTorch tensors (.pt files) in the cache directory.

Sources: src/piper/train/vits/dataset.py:175-200

Summary of Phoneme ID Ranges

The PHONEME_TO_ID map organizes the voice symbols into logical blocks:

Block	Range of IDs	Count
Meta (PAD, BOS, EOS)	0 - 2	3
Initials (including Ø)	3 - 26	24
Finals (a, o, e, etc.)	27 - 63	37
Tones (1 - 5)	64 - 68	5
Punctuation / Pauses	69 - 72	4

Sources: src/piper/phonemize_chinese.py:45-113

Chinese Phonemization provides a robust bridge between natural Chinese script and the neural synthesis engine. By utilizing g2pW for accurate polyphone resolution and unicode-rbnf for text normalization, the system ensures high-quality Mandarin speech output. Its specialized padding logic and syllable decomposition are tailored specifically to the phonetic structure of the Chinese language.