Relevant source files

The following files were used as context for generating this wiki page: - [libpiper/include/piper.h](https://github.com/javijupra/piper1-gpl/blob/main/libpiper/include/piper.h) - [libpiper/src/piper.cpp](https://github.com/javijupra/piper1-gpl/blob/main/libpiper/src/piper.cpp) - [libpiper/include/piper_impl.hpp](https://github.com/javijupra/piper1-gpl/blob/main/libpiper/include/piper_impl.hpp) - [libpiper/CMakeLists.txt](https://github.com/javijupra/piper1-gpl/blob/main/libpiper/CMakeLists.txt) - [libpiper/README.md](https://github.com/javijupra/piper1-gpl/blob/main/libpiper/README.md)

# Libpiper C++ API Libpiper is a shared library providing a C-style interface for the Piper text-to-speech engine. It enables high-performance, local neural speech synthesis by embedding [espeak-ng](https://github.com/espeak-ng/espeak-ng) for phonemization and using the [ONNX Runtime](https://github.com/microsoft/onnxruntime) for model inference. The API is designed to handle text-to-speech tasks in a streaming fashion, allowing developers to retrieve audio chunks as they are generated rather than waiting for the entire synthesis process to complete. Sources: [libpiper/README.md:1-5](), [libpiper/include/piper.h:18-20]() ## Core Architecture and Data Flow The architecture of `libpiper` centers around the `piper_synthesizer` structure, which manages the lifecycle of the speech synthesis process. This includes initializing the eSpeak-ng engine, loading ONNX voice models, and managing the internal queue of phonemes and IDs. ### Synthesis Lifecycle The synthesis process follows a specific state machine: 1. **Creation**: Initialize the environment and load the model. 2. **Start**: Phonemize the input text and populate the internal work queue. 3. **Iteration**: Retrieve synthesized audio chunks sequentially until the queue is exhausted. 4. **Cleanup**: Release resources and terminate the engine. The following diagram illustrates the high-level data flow from input text to audio output: ```mermaid flowchart TD A[Input Text] --> B[espeak-ng Phonemization] B --> C[Phoneme to ID Mapping] C --> D[Phoneme ID Queue] D --> E[ONNX Runtime Inference] E --> F[Audio Chunk Output] F -- "More chunks?" --> D F -- "Finished" --> G[End] ``` Sources: [libpiper/src/piper.cpp:115-240](), [libpiper/include/piper_impl.hpp:51-64]() ### Component Relationship Libpiper interacts with external libraries to perform specialized tasks. The relationship between the C API, the internal C++ implementation, and external dependencies is shown below: ```mermaid classDiagram class piper_h { <> +piper_create() +piper_synthesize_start() +piper_synthesize_next() +piper_free() } class piper_synthesizer { <> +session: Ort::Session +phoneme_id_queue: queue +espeak_voice: string } class espeak_ng { <> +espeak_Initialize() +espeak_TextToPhonemesWithTerminator() } class onnxruntime { <> +Run() } piper_h ..> piper_synthesizer : manages piper_synthesizer --> espeak_ng : uses for phonemes piper_synthesizer --> onnxruntime : uses for inference ``` Sources: [libpiper/include/piper.h:23-25](), [libpiper/include/piper_impl.hpp:51-78](), [libpiper/src/piper.cpp:32-40]() ## API Reference ### Data Structures The API utilizes three primary structures to manage state, audio data, and configuration. | Structure | Description | | :--- | :--- | | `piper_synthesizer` | Opaque handle containing the internal state of the TTS engine, including ONNX sessions and eSpeak configurations. | | `piper_audio_chunk` | Contains raw float samples, sample rate, and alignment data for a segment of synthesized speech. | | `piper_synthesize_options` | Controls synthesis parameters like speaker ID, speech speed (length scale), and noise levels. | Sources: [libpiper/include/piper.h:23-140]() ### Synthesis Options The `piper_synthesize_options` structure allows for fine-tuning the voice output. | Parameter | Type | Default | Description | | :--- | :--- | :--- | :--- | | `speaker_id` | `int` | 0 | The ID of the speaker to use for multi-speaker models. | | `length_scale` | `float` | 1.0 | Controls speed: < 1.0 is faster, > 1.0 is slower. | | `noise_scale` | `float` | 0.667 | Controls the amount of noise added during synthesis. | | `noise_w_scale` | `float` | 0.8 | Controls the variance in phoneme lengths. | Sources: [libpiper/include/piper.h:111-140](), [libpiper/include/piper_impl.hpp:35-37]() ## Implementation Details ### Phonemization and ID Mapping When `piper_synthesize_start` is called, the library uses eSpeak-ng to convert UTF-8 text into IPA phonemes. These phonemes are then mapped to integer IDs using a `phoneme_id_map` loaded from the voice's `.json` configuration file. The implementation automatically handles sentence terminators (periods, commas, etc.) and filters out language switch flags. ```cpp // Logic for handling sentence terminators in piper.cpp if (terminator == CLAUSE_PERIOD) { terminator_str = "."; } else if (terminator == CLAUSE_QUESTION) { terminator_str = "?"; } // ... sentence_phonemes[current_idx] += terminator_str; ``` Sources: [libpiper/src/piper.cpp:142-164](), [libpiper/src/piper.cpp:180-210]() ### Audio Inference Synthesis occurs in `piper_synthesize_next`. This function pops a set of phoneme IDs from the internal queue and runs them through the ONNX model. ```mermaid sequenceDiagram participant App as Application participant Lib as libpiper participant ONNX as ONNX Runtime App->>Lib: piper_synthesize_next(synth, &chunk) Lib->>Lib: Pop IDs from queue Lib->>ONNX: Run(input_ids, scales, speaker_id) ONNX-->>Lib: Float audio tensor + alignments Lib->>Lib: Copy data to chunk structure Lib-->>App: Return PIPER_OK / PIPER_DONE ``` Sources: [libpiper/src/piper.cpp:242-320]() ### Building and Dependencies The library requires `espeak-ng-data` to be present at runtime. The build system, managed via CMake, handles the external project integration for eSpeak-ng and the acquisition of ONNX Runtime shared libraries. ```cmake # From libpiper/CMakeLists.txt ExternalProject_Add(espeak_ng_external GIT_REPOSITORY https://github.com/espeak-ng/espeak-ng.git # ... ) target_link_libraries(piper ${ESPEAKNG_STATIC_LIB} ${UCD_STATIC_LIB} onnxruntime ) ``` Sources: [libpiper/CMakeLists.txt:28-40](), [libpiper/CMakeLists.txt:115-125]() ## Conclusion The Libpiper C++ API provides a robust bridge between high-level applications and the underlying neural TTS models. By abstracting the complexities of eSpeak-ng phonemization and ONNX inference into a simple "start-next-free" workflow, it facilitates the integration of local, high-quality speech synthesis into diverse software environments.