Browser Integration

The @speech-sdk/core is a universal TypeScript SDK designed to run seamlessly in Browser, Node.js, and Edge environments. It provides a unified interface for interacting with 13 different Text-to-Speech (TTS) providers, abstracting away the complexities of browser-specific audio handling, such as ReadableStream consumption and base64 audio decoding.

Sources: skills/speech-sdk/SKILL.md:15, README.md:10, AGENTS.md:17

Core Architecture for Browsers

The SDK uses a provider-agnostic architecture where a single public entry point, generateSpeech(), resolves into specific provider logic. In a browser context, the SDK leverages standard Web APIs like fetch and ReadableStream to manage audio data transfer and playback preparation.

Execution Flow

The following diagram illustrates how a TTS request is processed from a browser-based application through the SDK to a provider.

A visual representation of the internal call stack from the public API to the network layer.
Sources: AGENTS.md:19-21, src/providers/google/index.ts:161-171

Browser-Compatible Audio Formats

The SDK handles various audio media types commonly used in web browsers. It specifically provides utilities to wrap raw PCM (Pulse Code Modulation) data into a WAV container, ensuring that audio returned by providers like Google Gemini is immediately playable via standard HTML5 <audio> elements or the Web Audio API.

Sources: skills/speech-sdk/SKILL.md:25-27, src/providers/google/index.ts:187-195, src/conversation/pcm-concat.ts:250-256

Streaming in the Browser

The streamSpeech() function is the primary mechanism for real-time audio in the browser. It returns a ReadableStream<Uint8Array>, which can be directly piped into a browser's Response object or consumed chunk-by-chunk for low-latency playback.

Streaming Sequence

The sequence below shows how a browser client consumes a speech stream.

The flow of asynchronous audio chunks from the provider to the browser's readable stream.
Sources: README.md:104-110, src/providers/google/index.ts:219-228, src/tests/google-stream.test.ts:7-17

Multi-Speaker Conversations (Stitch Path)

When using generateConversation() in the browser for providers that do not support native dialogue, the SDK performs "Local Stitching." This involves:

1. Parallel Dispatch: Fetching turns in parallel using browser fetch.
2. PCM Decoding: Converting various formats into 16-bit Mono PCM.
3. Normalization: Applying RMS-leveling (default -20 dBFS) to ensure consistent volume.
4. Concatenation: Inserting gapMs silence between speakers and returning a single audio/wav file.

Sources: skills/speech-sdk/SKILL.md:43-52, src/conversation/pcm-concat.ts:182-192, src/conversation/pcm-concat.ts:216-224

Browser Integration Configuration

Developers can customize the browser's networking behavior by passing a custom fetch implementation or AbortSignal for request cancellation.

import { generateSpeech } from "@speech-sdk/core";

const controller = new AbortController();

const result = await generateSpeech({
  model: "openai/gpt-4o-mini-tts",
  text: "Browser integration test.",
  voice: "alloy",
  abortSignal: controller.signal, // Browser AbortSignal
  headers: { "X-Custom-Header": "value" } // Custom Browser Headers
});

Sources: skills/speech-sdk/SKILL.md:86-90, src/providers/google/index.ts:163-170

Summary

Browser integration with @speech-sdk/core is achieved through the use of standard Web APIs. The SDK ensures cross-provider compatibility by normalizing audio outputs into browser-friendly formats (WAV/MP3) and providing a consistent ReadableStream interface for streaming, while also handling complex tasks like audio stitching and volume normalization entirely within the client runtime.