Testing & Validation
Relevant source files
The following files were used as context for generating this wiki page:
Testing and validation within the Wikispeech server ecosystem ensure the integrity of the Text-to-Speech (TTS) pipeline, ranging from individual component configuration to end-to-end API functionality. The project utilizes a combination of shell scripts for environment orchestration, Python-based unit tests for core logic, and API integration tests to validate server responses and third-party service connections.
The scope of testing covers configuration validation, component loading, lexicon lookups, text processing, and audio synthesis. Developers are required to run local builds and tests before submitting pull requests to ensure compatibility with various microservices like MaryTTS, Pronlex, and Symbolset.
Sources: README.md:55-58, wikispeech_server/wikispeech.py:1000-1005
Build and Environment Orchestration
The project provides a comprehensive build script, build_and_test.sh, which automates the setup and testing of the entire Wikispeech stack. This script mimics the continuous integration environment and is used to validate that all microservices are correctly configured and reachable.
Build Workflow Logic
The build script performs the following high-level operations:
- Process Management: Identifies and kills existing instances of Wikispeech-related services to ensure a clean state.
- Dependency Setup: Clones and configures required repositories including
symbolset,pronlex, andmarytts. - Service Initialization: Starts background services with designated sleep intervals to allow for full initialization.
- Verification: Validates that all servers are running and terminates them upon completion.
The diagram above illustrates the sequential dependency management and service startup performed by the build orchestration script.
Sources: build_and_test.sh:45-103
API Integration Testing
The test_api.py suite validates the Flask-based REST API. It uses a test_client to simulate HTTP requests and asserts the structure and content of JSON responses.
Core API Test Cases
The API tests are categorized into functional groups:
- Discovery: Validates the
/optionsand/languagesendpoints to ensure supported languages (e.g., "en", "sv") are correctly advertised. - Text Processing: Tests the
/textprocessing/endpoint with various inputs (text, SSML) and validates the resulting JSON markup. - Synthesis: Tests the
/synthesis/endpoint by providing markup and verifying that an audio URL or audio data is returned along with token timings.
Sources: wikispeech_server/test_api.py:30-50, wikispeech_server/test_api.py:130-150, wikispeech_server/test_api.py:280-300
| Endpoint | Method | Purpose | Validation Criteria |
|---|---|---|---|
/ | OPTIONS | API Parameters | Returns JSON with allowed languages and parameters |
/textprocessing/ | GET/POST | Tokenization | Validates JSON structure containing paragraphs/sentences |
/synthesis/ | GET/POST | Audio Generation | Returns audio URL/data and token timing list |
/languages | GET | Language Support | Returns a list of strings for supported languages |
Sources: wikispeech_server/test_api.py:20-25, wikispeech_server/test_api.py:165-175, wikispeech_server/test_api.py:315-325
Voice and Configuration Validation
The system performs rigorous validation of voice configurations and text processor settings. This is handled both during server startup and via specialized test scripts.
Voice Class Unit Tests
The test_voice.py script uses the unittest framework to validate the Voice class logic. It ensures that:
- New voices can be initialized with valid configurations.
- Broken voice names or invalid mapper settings trigger a
VoiceException. - The
Mapperclient is correctly instantiated for voices requiring symbol set mapping.
Sources: wikispeech_server/test_voice.py:15-45
Dynamic Configuration Testing
The test_voice_configs.py script dynamically iterates through all registered languages, text processors, and voices to perform exhaustive synthesis tests.
This diagram represents the nested validation loop used to ensure every configured component in the system is functional.
Sources: wikispeech_server/test_voice_configs.py:35-70
Internal System Tests
The main server implementation (wikispeech.py) contains internal test functions that can be triggered to verify connectivity to external dependencies like MaryTTS and Lexicon servers.
test_config(): Verifies that the temporary audio directory exists and is writable, and checks for the presence of mandatory service URLs in the configuration.test_lexicon_client(): Performs a sample lookup ("apa", "hund") against the configured lexicon server and asserts that the returned transcriptions match expected values.test_textproc(): Sends a sample string through the default text processor to ensure the text processing pipeline is operational.
Sources: wikispeech_server/wikispeech.py:1008-1030, wikispeech_server/wikispeech.py:1050-1065, wikispeech_server/wikispeech.py:1135-1150
Conclusion
Testing and validation in the Wikispeech server project are structured to handle the complexity of a microservice-dependent architecture. By combining automated build scripts, API integration tests, and component-level unit tests, the system ensures that changes to configurations or adapters do not break the core TTS pipeline. Successful completion of the build_and_test.sh script is the primary gatekeeper for maintaining system stability across releases.