Libpiper C++ API

Relevant source files

The following files were used as context for generating this wiki page:

• libpiper/include/piper.h
• libpiper/src/piper.cpp
• libpiper/include/piper_impl.hpp
• libpiper/CMakeLists.txt
• libpiper/README.md

Libpiper is a shared library providing a C-style interface for the Piper text-to-speech engine. It enables high-performance, local neural speech synthesis by embedding espeak-ng for phonemization and using the ONNX Runtime for model inference.

The API is designed to handle text-to-speech tasks in a streaming fashion, allowing developers to retrieve audio chunks as they are generated rather than waiting for the entire synthesis process to complete.

Sources: libpiper/README.md:1-5, libpiper/include/piper.h:18-20

Core Architecture and Data Flow

The architecture of libpiper centers around the piper_synthesizer structure, which manages the lifecycle of the speech synthesis process. This includes initializing the eSpeak-ng engine, loading ONNX voice models, and managing the internal queue of phonemes and IDs.

Synthesis Lifecycle

The synthesis process follows a specific state machine:

1. Creation: Initialize the environment and load the model.
2. Start: Phonemize the input text and populate the internal work queue.
3. Iteration: Retrieve synthesized audio chunks sequentially until the queue is exhausted.
4. Cleanup: Release resources and terminate the engine.

The following diagram illustrates the high-level data flow from input text to audio output:

Component Relationship

Libpiper interacts with external libraries to perform specialized tasks. The relationship between the C API, the internal C++ implementation, and external dependencies is shown below:

API Reference

Data Structures

The API utilizes three primary structures to manage state, audio data, and configuration.

Structure	Description
piper_synthesizer	Opaque handle containing the internal state of the TTS engine, including ONNX sessions and eSpeak configurations.
piper_audio_chunk	Contains raw float samples, sample rate, and alignment data for a segment of synthesized speech.
piper_synthesize_options	Controls synthesis parameters like speaker ID, speech speed (length scale), and noise levels.

Sources: libpiper/include/piper.h:23-140

Synthesis Options

The piper_synthesize_options structure allows for fine-tuning the voice output.

Parameter	Type	Default	Description
speaker_id	int	0	The ID of the speaker to use for multi-speaker models.
length_scale	float	1.0	Controls speed: < 1.0 is faster, > 1.0 is slower.
noise_scale	float	0.667	Controls the amount of noise added during synthesis.
noise_w_scale	float	0.8	Controls the variance in phoneme lengths.

Sources: libpiper/include/piper.h:111-140, libpiper/include/piper_impl.hpp:35-37

Implementation Details

Phonemization and ID Mapping

When piper_synthesize_start is called, the library uses eSpeak-ng to convert UTF-8 text into IPA phonemes. These phonemes are then mapped to integer IDs using a phoneme_id_map loaded from the voice's .json configuration file. The implementation automatically handles sentence terminators (periods, commas, etc.) and filters out language switch flags.

// Logic for handling sentence terminators in piper.cpp
if (terminator == CLAUSE_PERIOD) {
    terminator_str = ".";
} else if (terminator == CLAUSE_QUESTION) {
    terminator_str = "?";
}
// ...
sentence_phonemes[current_idx] += terminator_str;

Sources: libpiper/src/piper.cpp:142-164, libpiper/src/piper.cpp:180-210

Audio Inference

Synthesis occurs in piper_synthesize_next. This function pops a set of phoneme IDs from the internal queue and runs them through the ONNX model.

Building and Dependencies

The library requires espeak-ng-data to be present at runtime. The build system, managed via CMake, handles the external project integration for eSpeak-ng and the acquisition of ONNX Runtime shared libraries.

# From libpiper/CMakeLists.txt
ExternalProject_Add(espeak_ng_external
    GIT_REPOSITORY https://github.com/espeak-ng/espeak-ng.git
    # ...
)
target_link_libraries(piper
    ${ESPEAKNG_STATIC_LIB}
    ${UCD_STATIC_LIB}
    onnxruntime
)

Sources: libpiper/CMakeLists.txt:28-40, libpiper/CMakeLists.txt:115-125

Conclusion

The Libpiper C++ API provides a robust bridge between high-level applications and the underlying neural TTS models. By abstracting the complexities of eSpeak-ng phonemization and ONNX inference into a simple "start-next-free" workflow, it facilitates the integration of local, high-quality speech synthesis into diverse software environments.