High-Level Architecture
Relevant source files
The following files were used as context for generating this wiki page:
The Wikispeech server is a Python-based middleware designed to facilitate user-generated text-to-speech (TTS) synthesis, particularly for Wikipedia. It acts as a central hub that orchestrates the flow of data between frontend clients, text processing components, and various synthesis engines.
The architecture is built upon a microservices approach, leveraging Flask for its API layer and a plugin-based adapter system to integrate third-party TTS technologies such as MaryTTS, Piper, and Flite. The system follows a two-stage pipeline: text processing (tokenization and transcription) followed by speech synthesis (audio generation and timing synchronization).
Sources: README.md, wikispeech_server/wikispeech.py:1-100, wikispeech_server/adapters/README.md
System Overview
The Wikispeech server manages the lifecycle of a TTS request from raw text input to an audio response containing timing data for word highlighting.
The diagram above illustrates the high-level data flow from the initial user request through the processing and synthesis stages.
Sources: wikispeech_server/wikispeech.py:230-350, wikispeech_server/adapters/README.md
Core Components
1. API Layer (Flask)
The server exposes a RESTful API that handles requests for supported languages, available voices, and the synthesis pipeline itself. It supports both GET and POST methods, accepting raw text or SSML (Speech Synthesis Markup Language).
| Endpoint | Method | Description |
|---|---|---|
/ | GET/POST | Primary synthesis endpoint. |
/languages | GET | Returns a list of supported languages. |
/textprocessing | GET/POST | Returns intermediate tokenized markup. |
/synthesis | GET/POST | Direct access to the synthesis stage using pre-processed markup. |
/version | GET | Returns application build and release info. |
Sources: wikispeech_server/wikispeech.py:70-150, wikispeech_server/wikispeech.py:420-450
2. Text Processing Pipeline
The text processing stage converts input text into a structured "utterance" (UTT) format. This format is a JSON-like dictionary containing paragraphs, sentences, phrases, and tokens with linguistic metadata.
- Tokenization: Splitting text into individual words and punctuation.
- G2P (Grapheme-to-Phoneme): Converting orthographic text into phonetic transcriptions.
- SSML Mapping: Handling standard SSML tags like
<phoneme>and<break>.
Sources: wikispeech_server/wikispeech.py:480-550, wikispeech_server/adapters/piper_adapter.py:40-65
3. Adapter System
The server uses an adapter pattern to remain engine-agnostic. Adapters enable Wikispeech to communicate with third-party software by implementing specific interfaces.
- Textprocessing Adapters: Must implement a function defined by the
callparameter in the configuration (e.g.,marytts_preproc). - Synthesis Adapters: Must implement a
synthesisefunction that returns a pointer to generated audio and a list of tokens with timing information.
Sources: wikispeech_server/adapters/README.md, wikispeech_server/adapters/flite_adapter.py:20-50
Data Flow: Synthesis Request
When a synthesis request is received, the server executes the following sequence:
The sequence diagram demonstrates the transformation of data from raw text into a synchronized audio response.
Sources: wikispeech_server/wikispeech.py:230-300, wikispeech_server/wikispeech.py:650-700
Audio Management and Conversion
Wikispeech prioritizes modern, efficient audio formats.
- Opus Encoding: All generated WAV files from underlying engines are converted to Opus format using the
opusencsystem tool. - Base64 Delivery: To reduce client-side latency and bypass complex proxy configurations for static files, audio can be delivered as a Base64-encoded string within the JSON response.
- Timing Synchronization: Adapters provide
endtimevalues for tokens. The server ensures these are converted to milliseconds before delivery to the client.
Sources: wikispeech_server/wikispeech.py:45-55, wikispeech_server/wikispeech.py:730-760, wikispeech_server/wikispeech.py:880-910
Frontend Integration: The Simple Player
The wikispeech_simple_player.js utility provides a reference implementation for consuming the API. It manages:
- Request Handling: Sending text/SSML to the server via AJAX.
- Audio Playback: Loading Base64 or URL-based audio into HTML5
<audio>elements. - Read-Along Highlighting: Using the
tokensarray (containingorth,begin, anddur) to highlight words in the DOM as they are spoken.
Sources: wikispeech_simple_player.js:100-150, wikispeech_simple_player.js:350-400
Summary
The Wikispeech server architecture is a robust middleware solution that abstracts the complexities of various TTS engines into a unified JSON API. By utilizing a pipeline of configurable textprocessors and synthesis adapters, it provides a flexible environment for linguistic processing and high-quality audio delivery suitable for interactive web applications.