Hume Provider Integration
Relevant source files
The following files were used as context for generating this wiki page:
The Hume provider integration in the Speech SDK enables developers to leverage Hume AI's expressive text-to-speech (TTS) models, specifically the octave-1 and octave-2 series. This integration provides a standardized interface for generating single-turn speech, streaming audio, and multi-turn native dialogues. It supports advanced features such as word-level timestamps (alignment), voice cloning by name, and expressive steering through custom descriptions.
Architecturally, the integration encapsulates the complexity of Hume's multi-endpoint API, automatically routing requests between the high-performance /v0/tts/file endpoint for raw audio and the JSON-based /v0/tts endpoint when metadata like word alignment is required.
Sources: src/providers/hume/index.ts:16-45, skills/speech-sdk/references/providers/hume.md:1-20
Core Architecture and Models
The integration is centered around the HumeSpeechProvider class, which implements the SpeechProvider interface. It manages authentication using the X-Hume-Api-Key header and communicates with the Hume V0 API.
Supported Models
| Model | ID | Features | Default |
|---|---|---|---|
| Octave 2 | octave-2 | Streaming, Inline Voice Cloning, Native Timestamps | Yes |
| Octave 1 | octave-1 | Streaming | No |
Sources: src/providers/hume/index.ts:21-45, skills/speech-sdk/references/providers/hume.md:10-15
Request Flow Overview
The provider determines the target API endpoint based on the requested model and the need for timestamps.
This diagram shows how the SDK switches between Hume's JSON and binary endpoints depending on whether metadata is requested.
Sources: src/providers/hume/index.ts:101-140, src/tests/hume-timestamps.test.ts:121-140
Generation and Streaming
The SDK supports both standard generation (returning a complete buffer) and streaming (returning a ReadableStream).
Single Utterance Generation
For standard generation, the provider wraps the text and voice into an utterances array. If a voice is provided, it is formatted as an object with provider: "HUME_AI".
// Internal request body structure
const body = {
version: "2", // Mapped from octave-2
utterances: [
{
text: "Hello",
voice: { name: "Kora", provider: "HUME_AI" }
}
]
};
Sources: src/providers/hume/index.ts:89-98, src/tests/hume-speech-model.test.ts:51-64
Streaming
Streaming requests are routed to the /v0/tts/stream/file endpoint. This allows for low-latency audio delivery by processing the response body as a ReadableStream.
Sources: src/providers/hume/index.ts:167-200, src/tests/hume-stream.test.ts:5-25
Word-Level Timestamps (Alignment)
The octave-2 model supports native word alignment. When timestamps: "on" or "auto" is requested, the SDK performs the following:
- Endpoint Switch: Routes the request to
/v0/ttsinstead of/v0/tts/file. - Configuration: Injects
include_timestamp_types: ["word"]andsplit_utterances: falseinto the request body. - Parsing: Extracts snippets from the JSON response and flattens them into the SDK's
WordTimestamp[]format, converting milliseconds to seconds.
Native Dialogue Support
Hume supports native multi-turn dialogue within a single API call. This is utilized by generateConversation when all turns use the Hume provider.
- Logic: The SDK maps multiple conversation turns into the
utterancesarray in a single request. - Stitching: For cross-provider conversations, the provider declares
getStitchOptionsreturningaudio/pcm;rate=48000. Hume's PCM output is always 48 kHz mono 16-bit. - Capabilities: The integration supports a minimum of 1 voice and a conservative maximum of 4 voices for native dialogue.
Sources: src/providers/hume/index.ts:219-270, src/tests/e2e/conversation-hume.e2e.test.ts:12-25
Configuration and Integration
API Parameters
| Parameter | Header / Key | Description |
|---|---|---|
| API Key | X-Hume-Api-Key | Required for authentication. |
| Base URL | baseURL | Defaults to https://api.hume.ai/v0. |
| User Agent | X-User-Agent | SDK identifier for telemetry. |
| Description | description | Passed via providerOptions for expressive steering. |
Sources: src/providers/hume/index.ts:60-65, src/tests/hume-speech-model.test.ts:35-45, skills/speech-sdk/references/providers/hume.md:43-52
Factory Function
The createHume factory simplifies instantiation, allowing for API key injection via configuration or environment variables (HUME_API_KEY).
import { createHume } from "@speech-sdk/core/hume";
const hume = createHume({ apiKey: "your-key" });
// Usage with generateSpeech
const result = await generateSpeech({
model: hume("octave-2"),
text: "Expressive audio",
voice: "Kora"
});
Sources: src/providers/hume/index.ts:275-285, skills/speech-sdk/references/providers/hume.md:58-62
The Hume Provider Integration is a robust module that prioritizes octave-2 capabilities while maintaining backward compatibility for octave-1. By handling the nuances of Hume's dual-endpoint system and multi-speaker utterances, it provides a seamless experience for generating high-quality, expressive AI speech with precise word timing.