Audio Playback System

Relevant source files

The following files were used as context for generating this wiki page:

• src/piper/__main__.py
• src/piper/voice.py
• src/piper/http_server.py
• libpiper/include/piper.h
• libpiper/src/piper.cpp
• CHANGELOG.md

The Audio Playback System in Piper is responsible for taking synthesized audio data—typically generated as raw PCM buffers or 16-bit integer samples—and either outputting it to a physical audio device, writing it to a standard file format, or streaming it via network protocols. The system is designed to handle audio in chunks, allowing for low-latency streaming and real-time playback during the synthesis process.

At a high level, the system supports three primary output modes: direct playback using external utilities like ffplay, raw PCM streaming to stdout, and structured file output using the WAVE format. It manages audio characteristics such as sample rate, bit depth, and channel count to ensure compatibility between the neural synthesis engine and the target output device.

Audio Data Structures

The system represents audio data using specialized structures that contain both the raw signal and metadata required for correct playback or encoding.

Python AudioChunk

In the Python implementation, audio is encapsulated in the AudioChunk dataclass. This structure maintains the audio in multiple formats (float32 and int16) to support various processing and output requirements.

Field	Type	Description
sample_rate	int	Rate of chunk samples in Hertz (e.g., 22050).
sample_width	int	Width of chunk samples in bytes (typically 2 for 16-bit).
sample_channels	int	Number of audio channels (typically 1 for mono).
audio_float_array	np.ndarray	Raw audio data as float32 in the range [-1, 1].
audio_int16_bytes	bytes	Audio data converted to signed 16-bit PCM bytes.

Sources: src/piper/voice.py:44-71, src/piper/voice.py:92-101

C API Audio Chunk

The C implementation (libpiper) uses a piper_audio_chunk struct, which provides pointers to raw float samples and alignment metadata for integration with C-based applications.

The alignments field in the C structure represents the audio sample count for each phoneme id, allowing for synchronized visual feedback during playback.
Sources: libpiper/include/piper.h:25-86

Playback and Output Modes

The system determines the output method based on command-line arguments and environment availability.

Direct Hardware Playback

If no output file is specified and the ffplay utility is detected on the system, Piper utilizes the AudioPlayer class to stream synthesized chunks directly to the hardware.

WAVE File Encoding

For persistent storage, the system uses the wave module to write structured files. It automatically configures the WAV headers based on the first synthesized chunk's metadata.

Parameter	Source Value
Frame Rate	audio_chunk.sample_rate
Sample Width	audio_chunk.sample_width
Channels	audio_chunk.sample_channels

Sources: src/piper/__main__.py:139-148, src/piper/voice.py:346-360

Streaming and HTTP Output

The http_server.py module facilitates network-based audio delivery. It synthesizes text and returns a complete WAVE file as a byte stream in response to HTTP POST requests.

Audio Normalization and Processing

Before audio is passed to the playback or output components, it undergoes several transformation steps to ensure quality and consistency.

1. Normalization: If enabled in SynthesisConfig, the audio is normalized by dividing the signal by its maximum absolute value to reach peak volume without clipping.
2. Volume Adjustment: A volume multiplier is applied (default 1.0) to scale the signal.
3. Clipping: The signal is clipped to the range [-1.0, 1.0] before conversion to 16-bit integers.
4. Silence Insertion: During multi-sentence synthesis, the system calculates and inserts 16-bit silence buffers between sentences based on the --sentence-silence parameter.

Sources: src/piper/voice.py:293-305, src/piper/__main__.py:133-136, libpiper/src/piper.cpp:430-435

Conclusion

The Piper Audio Playback System provides a flexible interface for consuming neural TTS output. By abstracting audio chunks into standardized structures across Python and C, it enables seamless transitions between real-time local playback, file-based archival, and high-performance streaming via HTTP. The system's ability to handle dynamic sample rates and insert configurable silences makes it adaptable to various voice models and user requirements.