SSML Handling
Relevant source files
The following files were used as context for generating this wiki page:
SSML (Speech Synthesis Markup Language) handling within the Wikispeech server allows for fine-grained control over Text-to-Speech (TTS) synthesis. It enables the system to process structured XML input containing specific speech instructions such as phonetic transcriptions, breaks, and prosody adjustments. The server supports SSML as an alternative to plain text for both its textprocessing and synthesis API endpoints.
Sources: wikispeech_server/wikispeech.py:240-250, wikispeech_server/options.py:28-33
Architecture and Data Flow
The Wikispeech server acts as a coordinator, receiving SSML input and routing it through text processors and voice adapters. When input_type is set to ssml, the server bypasses or modifies standard tokenization logic to respect the markup tags provided in the input.
SSML Processing Pipeline
The following diagram illustrates the flow of an SSML request through the system components:
The system first identifies the input type. If SSML is detected, it may perform alphabet mapping (e.g., converting IPA to X-SAMPA) before passing the structured data to specific backend adapters like MaryTTS, Flite, or eSpeak.
Sources: wikispeech_server/wikispeech.py:480-500, wikispeech_server/adapters/marytts_adapter.py:55-70
Key Components and Functions
API Endpoints
The server exposes SSML capabilities through two primary interfaces defined in options.py:
| Endpoint | Parameter | Allowed Values | Description |
|---|---|---|---|
/textprocessing | input_type | text, ssml | Determines if input is plain text or SSML markup. |
/synthesis | input_type | markup | Internal structure derived from processed SSML. |
/ (Main API) | input_type | text, ssml, ipa | High-level synthesis entry point. |
Sources: wikispeech_server/options.py:28-33, wikispeech_server/options.py:100-105
IPA Mapping Logic
The server provides a specific utility function mapIpaInput to handle SSML inputs that use the International Phonetic Alphabet (IPA). It detects <phoneme alphabet="ipa"> tags and uses a mapper service to convert them to X-SAMPA, which is more commonly supported by underlying synthesis engines.
Sources: wikispeech_server/wikispeech.py:545-570
Adapter Implementations
Different adapters handle SSML differently based on their backend engine capabilities:
- MaryTTS Adapter: Converts internal Wikispeech utterance structures back into
MaryXML(a variant of SSML) for synthesis. It specifically handles<prosody>and<boundary>tags. - Flite Adapter: Utilizes a
utt2ssmlfunction to generate SSML strings for the Flite engine, mapping phonetic strings to the Flite-specific phoneme set. - eSpeak Adapter: Provides
utt2ssmlandutt2phonemicsfunctions to translate structured text and phonemes into eSpeak-compatible formats.
Sources: wikispeech_server/adapters/marytts_adapter.py:115-130, wikispeech_server/adapters/flite_adapter.py:115-145, wikispeech_server/adapters/espeak_adapter.py:100-115
SSML Element Support
The system recognizes and processes several standard SSML elements and attributes:
| Element | Attribute(s) | Usage |
|---|---|---|
<speak> | xml:lang, version | Root element for SSML documents. |
<phoneme> | alphabet, ph | Used for custom phonetic transcriptions (e.g., X-SAMPA). |
<break/> | time, strength | Inserts silences or pauses in speech. |
<prosody> | pitch, range, rate | Controls the melodic and rhythmic aspects of speech. |
<sub/> | alias | Provides text substitution for the enclosed content. |
Sources: wikispeech_server/test_ssml.py:20-30, wikispeech_server/adapters/marytts_adapter.py:200-230, workflow_demo/test_workflow.js:255-265
Sequence of SSML Synthesis
The following sequence diagram shows the interaction between the client, the Wikispeech server, and the MaryTTS adapter during an SSML synthesis call:
Implementation Details
Transcription Validation
In the workflow_demo, SSML transcriptions are validated using the validateentry endpoint of the lexicon server. This ensures that the phonemes provided in the ph attribute of a <phoneme> tag conform to the allowed symbol set for the specific language (e.g., sv-se_ws-sampa).
Sources: workflow_demo/test_workflow.js:510-540
Formatting for Engines
Adapters often need to escape or transform SSML to meet engine requirements. For example, the marytts_adapter uses mapSsmlTranscriptionsToMary to ensure transcriptions do not contain characters that would break the XML parser, such as >.
Sources: wikispeech_server/adapters/marytts_adapter.py:155-175
Summary
SSML handling in Wikispeech Server provides a flexible way to override default text processing. By supporting standard tags like <phoneme> and <prosody>, and implementing robust mapping logic for alphabets like IPA, the server ensures high-quality, customizable speech synthesis across multiple backend engines.