Player & Read-Along Components

Relevant source files

The following files were used as context for generating this wiki page:

• wikispeech_simple_player.js
• wikispeech_server/static/read-along.js
• workflow_demo/test_workflow.js
• wikispeech_server/wikispeech.py
• wikispeech_server/templates/output.html
• wikispeech_server/options.py

The Player and Read-Along components in Wikispeech provide the frontend infrastructure necessary to synchronize synthesized audio with its corresponding text. These components facilitate interactive "speak" functionality where individual words are highlighted in real-time as they are played back by the audio engine. This system serves as the primary user interface for consuming Text-to-Speech (TTS) services within a web environment, supporting both plain text and SSML (Speech Synthesis Markup Language) inputs.

The architecture relies on a combination of JavaScript-driven timing synchronization on the client-side and detailed timing metadata (token start/end times) generated by the server-side synthesis adapters. By mapping these timing offsets to DOM elements, the system creates an immersive "read-along" experience where users can click on specific words to seek the audio or watch the text highlight automatically during playback.

Sources: wikispeech_simple_player.js, wikispeech_server/static/read-along.js, wikispeech_server/wikispeech.py:230-245

Simple Player Architecture

The wikispeech_simple_player.js script serves as the core coordinator for initiating synthesis requests and rendering the interactive audio player. It handles the transition from static HTML paragraphs to interactive, tokenized spans.

Component Logic and Data Flow

The player operates by extracting text from a DOM container, determining the language context, and sending a POST request to the Wikispeech server. Upon receiving a JSON response containing base64 audio and token timing metadata, it reconstructs the container's inner HTML with spans tagged with timing data.

The diagram shows the sequence from a user interaction to the final synchronized playback state.
Sources: wikispeech_simple_player.js:106-155, wikispeech_server/wikispeech.py:214-230

Key Synchronization Functions

Function	Description
play(id)	Initiates the synthesis request and handles the JSON response to setup the player.
addTimingInfoFromJson(container, info)	Iterates through server-provided tokens to create <span> elements with data-begin and data-dur attributes.
connectTimingAndAudio(root, audio)	Sets up event listeners on the audio element and text spans to handle highlighting and seeking.
toggleAudioControls()	Shows or hides the native HTML5 audio controls based on configuration.

Sources: wikispeech_simple_player.js:106, wikispeech_simple_player.js:180, wikispeech_simple_player.js:316

Read-Along Synchronization Logic

The read-along functionality is encapsulated in the ReadAlong object (found in read-along.js) and the connectTimingAndAudio function. This logic ensures that the speaking class is applied to the correct text segment based on the current currentTime of the audio element.

Real-time Highlighting Sequence

Synchronization is achieved by building an index of word times and using a combination of the timeupdate event and setTimeout for high-precision updates (sub-250ms resolution).

The sequence diagram illustrates how the JavaScript logic compensates for low-frequency timeupdate events using timers.
Sources: wikispeech_server/static/read-along.js:77-118, wikispeech_simple_player.js:316-335

Interactive Features

1. Click to Seek: Users can click any span containing a data-begin attribute. The system updates audio.currentTime to the value of data-begin plus a small offset (0.01s) to prevent early selection errors.
2. Double Click to Play: Double-clicking a text container triggers immediate playback.
3. Automatic Focus: The autofocus_current_word configuration allows the system to set focus on the currently spoken word, aiding accessibility.

Sources: wikispeech_server/static/read-along.js:145-165, wikispeech_simple_player.js:342-355

Server-Side Integration

The player components rely on specific API outputs from the wikispeech.py server. The server must provide a specific JSON structure for the frontend to function.

Token Data Model

The server returns a list of tokens, each containing orthography and a specific end time. The frontend calculates the duration by subtracting the previous token's end time.

# Example of server-side timing conversion to milliseconds
def convertTokenTimingsToMilliseconds(tokens):
    for token in tokens:
        if "endtime" in token:
            token["endtime"] = int(token["endtime"]*1000)
    return tokens

Sources: wikispeech_server/wikispeech.py:448-453

API Parameters for Synthesis

The player utilizes the following parameters when communicating with the synthesis endpoint:

Parameter	Type	Required	Description
input	String	Yes	The text or SSML to be synthesized.
lang	String	Yes	ISO 639-1 code (e.g., 'sv', 'en').
input_type	String	No	'text' or 'ssml'. Defaults to 'text'.
voice	String	No	Specific voice name (e.g., 'en_US-bryce-medium').
output_type	String	No	'json' or 'html'. Usually 'json' for AJAX players.

Sources: wikispeech_server/options.py:10-44, wikispeech_server/wikispeech.py:214-225

Workflow Demo & Lexicon Integration

The test_workflow.js file provides advanced player interactions used in development tools. It includes functionality to "play" specific lexicon entries or transcription strings before they are committed to the database.

Transcription Playback

When a user edits a transcription in the Lexicon Entry Editor, the system validates the Sampa/IPA string and generates a temporary SSML block to preview the audio.

function playSSML(trans, lang) {
    ssml = makeSSMLTranscription(trans, lang);
    container = document.createElement("p");
    container.setAttribute("lang", lang);
    container.setAttribute("class", "ssml");
    container.appendChild(ssml);
    // Player globals
    useOriginalText = false;
    showControls = false;
    play(container);
}

Sources: workflow_demo/test_workflow.js:462-478

SSML Parsing logic

The player handles specialized SSML tags by filtering them before synthesis. Specifically, tags related to ssml:s (sentences) or p (paragraphs) are processed to ensure the TTS engine receives valid markup.
Sources: workflow_demo/test_workflow.js:283-305, wikispeech_simple_player.js:90-103

Summary

The Player and Read-Along components form a tightly coupled system between the Python backend and JavaScript frontend. By leveraging HTML5 data attributes (data-begin, data-dur) and standard audio events, Wikispeech provides a responsive interface where text and speech are synchronized at the word level. This architecture supports standard user consumption via wikispeech_simple_player.js as well as administrative transcription testing through the workflow demo tools.

Sources: wikispeech_simple_player.js, wikispeech_server/static/read-along.js, workflow_demo/test_workflow.js