Captions (SRT / WebVTT)
Relevant source files
The following files were used as context for generating this wiki page:
The Captions module provides functionality to convert word-level timestamps generated during text-to-speech synthesis into industry-standard subtitle formats. It primarily supports SubRip (.srt) and WebVTT (.vtt) formats, enabling developers to synchronize spoken audio with visual text overlays in media players or web browsers via HTML <track> elements.
This system handles complex text processing, including typography normalization, HTML escaping for WebVTT, and intelligent cue splitting based on sentence boundaries, character limits, and duration constraints. It serves as a post-processing layer for the Word-Level Timestamps feature.
Sources: src/captions.ts:311-331, README.md:144-150
Architecture and Data Flow
The caption generation process follows a pipeline that transforms raw WordTimestamp data into a formatted string. The logic is encapsulated in the timestampsToCaptions function, which orchestrates text normalization, grouping, splitting, and final formatting.
Generation Pipeline
The following diagram illustrates the flow from word timestamps to the final caption file:
The pipeline ensures that text is cleaned and structured correctly before being formatted into the specific syntax required by SRT or WebVTT.
Sources: src/captions.ts:348-390, src/tests/captions.test.ts:202-210
Text Processing and Normalization
Before formatting, the SDK performs several sanitization steps to ensure compatibility across different subtitle parsers and regions.
Typography Normalization
The normalizeTypography function replaces non-ASCII characters with standard equivalents to prevent corruption in parsers that may not handle certain UTF-16 units correctly.
- Curly Quotes: Replaced with standard ASCII apostrophes (
') and double quotes ("). - Dashes: En-dashes and em-dashes are converted to hyphens (
-). - Control Characters: C0 control characters (except common whitespace) and
DELare stripped to prevent truncation. - Whitespace: Internal runs of multiple spaces or tabs are collapsed into a single space.
Sources: src/captions.ts:10-38, src/tests/captions.test.ts:70-112
WebVTT Escaping
For the WebVTT format, text is escaped to prevent conflict with internal WebVTT markup tags (like <v> or <b>).
&becomes&<becomes<>becomes>
Sources: src/captions.ts:46-60, src/tests/captions.test.ts:145-155
Segmentation Logic
The SDK uses a hierarchical approach to split text into "cues" (the blocks displayed on screen).
1. Sentence Grouping
Words are first grouped into sentences based on a SENTENCE_TERMINATOR regex. This regex supports multiple writing systems:
- ASCII:
.,!,? - CJK:
。,！,？ - Devanagari:
।,॥ - Arabic:
؟,۔
Sources: src/captions.ts:93-102, src/tests/captions.test.ts:114-170
2. Cue Subdividing
If a sentence is too long to fit in a single cue, it is subdivided by splitSentenceIntoCues using the following priority:
- Character Budget: If adding a word exceeds
maxCharsPerCue. - Duration Budget: If the cue length exceeds
maxCueDurationMs. - Soft Breaks: If a word ends with a comma-equivalent terminator and the current phrase is longer than
longPhraseCommaBreakChars.
Sources: src/captions.ts:159-197, src/tests/captions.test.ts:182-212
3. Line Wrapping
Within each cue, words are wrapped into lines using wrapCueText. It tries to keep each line under maxLineLength while respecting the maxLines per cue.
Sources: src/captions.ts:219-245, src/tests/captions.test.ts:220-244
Configuration and Formats
The timestampsToCaptions function accepts a CaptionsOptions object to customize the output.
Supported Formats
| Format | Signature | Timestamp Separator | Escaping |
|---|---|---|---|
| SRT | None | Comma (,) | Raw Text |
| WebVTT | WEBVTT | Period (.) | HTML Escaped |
Sources: src/captions.ts:253-261, src/tests/captions.test.ts:246-330
Configuration Options
| Option | Type | Default | Description |
|---|---|---|---|
format | CaptionFormat | "srt" | Target format: "srt" or "vtt". |
maxLineLength | number | 42 | Max characters per line. |
maxLinesPerCue | number | 2 | Max lines visible in one cue. |
maxCueDurationMs | number | 7000 | Max time a cue stays on screen. |
maxCharsPerCue | number | 84 | Max total characters (overrides lines * length). |
longPhraseCommaBreakChars | number | 60 | Char threshold for comma-based breaks. |
Sources: src/captions.ts:271-300, README.md:173-180
Implementation Example
The conversion is typically performed on the timestamps array returned by generateSpeech.
import { generateSpeech, timestampsToCaptions } from '@speech-sdk/core';
const result = await generateSpeech({
model: 'elevenlabs/eleven_v3',
text: 'Hello world. This is a test.',
voice: 'JBFqnCBsd6RMkjVDRZzb',
timestamps: 'on',
});
// Generate WebVTT for web players
const vtt = timestampsToCaptions(result.timestamps ?? [], {
format: 'vtt',
maxLineLength: 30
});
Sources: README.md:151-171, src/index.ts:3-4
Summary
The Captions module in the Speech SDK provides a robust utility for generating synchronized subtitle files from word-level timings. By handling typography normalization and intelligent segmentation across multiple languages, it ensures high-quality captioning output compatible with standard media workflows and web-based video players.