ElevenLabs API Integration
Relevant source files
The following files were used as context for generating this wiki page:
The voice-elevenlabs project is a plugin for Claude Code that integrates with the ElevenLabs v3 API to provide comprehensive voice capabilities. It enables users to convert text to natural-sounding speech, generate multi-speaker dialogues, create custom voices from text descriptions, and transcribe audio files. The integration is primarily achieved through a set of shell commands that wrap ElevenLabs API calls and a hook system for automated voice notifications.
This integration aims to add a layer of accessibility and interactivity to Claude Code, allowing for hands-free operation and more expressive, conversational interactions. It leverages features from ElevenLabs v3, such as emotion control via audio tags and advanced voice design.
Sources: README.md:5-6, agents/voice-assistant.md:3-4, skills/voice-responses.md:6-7
Core Components & Setup
The integration relies on two main external components and one environment variable for configuration.
Dependencies:
- ElevenLabs API Key: A free or paid API key from ElevenLabs is required for authentication.
claude-voice-notifications: An NPM package used to handle the playback of generated audio for notifications.
Configuration:
The primary configuration is setting the ELEVENLABS_API_KEY environment variable. The voice ID can also be configured by setting the ELEVENLABS_VOICE_ID variable.
# Set API Key
echo 'export ELEVENLABS_API_KEY="your_key_here"' >> ~/.zshrc
# Set Voice ID (Optional)
export ELEVENLABS_VOICE_ID="pNInz6obpgDQGcFmaJgB"
Sources: README.md:18-26, README.md:68-69, commands/voice-config.md:11
Commands Overview
The plugin exposes several commands to interact with the ElevenLabs API.
| Command | Description |
|---|---|
/speak [text] | Speak text with emotion tags [excited], [whispers], etc. |
/dialogue | Generate multi-speaker conversations. |
/create-voice | Create a custom voice from a text description. |
/transcribe | Convert audio to text using Scribe v2. |
/voices | List available ElevenLabs voices. |
/voice-config | View and modify voice settings. |
Sources: README.md:33-42
Text-to-Speech (TTS)
The core functionality is converting text into speech using various commands that leverage the ElevenLabs v3 model.
/speak Command
This command converts a given text string into speech. It supports special "Audio Tags" to add emotional nuance to the generated voice.
Usage:
/speak [excited] We did it! The tests passed!
The following diagram illustrates the sequence of executing the /speak command.
This diagram shows the user command triggering an API call to ElevenLabs, which returns an audio file that is then played back.
Sources: commands/speak.md, README.md:45-48
Audio Tags
The v3 model supports tags to control the emotion and delivery of the speech.
| Tag | Description |
|---|---|
[excited] | Enthusiastic |
[whispers] | Quiet, intimate |
[laughs] / [laughing] | Natural laughter |
[sighs] | Thoughtful pause |
[angry] | Intense |
[sad] | Melancholic |
[nervous] | Hesitant |
[shouting] | Loud |
[sarcastically] | Dry humor |
Sources: README.md:50-60, commands/speak.md:12-21
/dialogue Command
This command generates a conversation between multiple speakers. It supports natural interruptions, per-speaker emotional cues, and automatic pacing.
Format:
/dialogue "Alice: Hey, did you see the new feature?
Bob: [excited] Yes! It's incredible!"
Voices can be assigned to speakers, otherwise a default and complementary voices are used. Interruptions are denoted with a trailing em-dash (—).
Sources: commands/dialogue.md:6-18, README.md:62-66
/create-voice Command (Voice Design v3)
Users can generate a unique AI voice from a text description without providing audio samples. The command uses the ElevenLabs Voice Design v3 API to generate three voice candidates for the user to preview and select.
Example:
/create-voice "warm female narrator, mid-30s, slight British accent, professional"
Description Tips:
- Age:
young,middle-aged,elderly - Gender:
male,female,neutral - Accent:
British,Southern US,Australian - Tone:
warm,authoritative,playful - Quality:
clear,raspy,smooth,deep
Sources: commands/create-voice.md:6-21, README.md:68-71
Speech-to-Text (Transcription)
/transcribe Command (Scribe v2)
This command converts speech from an audio file (local or URL) into text using the ElevenLabs Scribe v2 model. It supports various audio formats, speaker diarization, and word-level timestamps.
Usage:
transcribe file:/path/to/audio.mp3transcribe url:https://example.com/audio.mp3
The command is designed to be used for meeting notes, transcribing voice memos, and generating captions.
Sources: commands/transcribe.md:6-20, README.md:73-76
Voice Management
/voices and /voice-config Commands
These commands allow users to manage their voice settings.
/voices: Lists available pre-made voices and provides acurlcommand to fetch all voices from a user's account./voice-config: Allows viewing the current configuration (voice ID, API key status) and setting a new default voice usingset-voice [voice_id].
Default Voices:
| Voice | ID | Style |
|---|---|---|
| George | JBFqnCBsd6RMkjVDRZzb | Natural male (default) |
| Adam | pNInz6obpgDQGcFmaJgB | Deep male |
| Rachel | 21m00Tcm4TlvDq8ikWAM | Natural female |
| Domi | AZnzlk1XvdvUeBnXmlld | Young female |
Sources: commands/voices.md:6-15, commands/voice-config.md:6-17, README.md:78-85
Automated Voice Hooks
The plugin uses a hook system to trigger voice notifications automatically on specific Claude Code events. The configuration for these hooks is defined in hooks/hooks.json.
| Event | Script/Command | Description |
|---|---|---|
Stop | hooks/scripts/stop-voice.sh | Announces "[project]: done" when Claude finishes a task. |
SubagentStop | hooks/scripts/subagent-voice.sh | Announces "[project]: [task]" when a subagent completes its task. |
Notification | claude-voice-notifications | Announces when Claude needs permission or input. |
Sources: hooks/hooks.json:2-17, README.md:87-92
The following diagram shows the general flow for event-triggered voice notifications.
This flowchart illustrates how different Claude events trigger specific scripts that format a message and send it for audio playback via the ElevenLabs API.
Sources: hooks/hooks.json, hooks/scripts/stop-voice.sh, hooks/scripts/subagent-voice.sh
The subagent-voice.sh script constructs a message by extracting the project name from the current working directory and the task description from the conversation transcript.
Sources: hooks/scripts/subagent-voice.sh:5-29
Agent and Skill Integration
The ElevenLabs functionalities are also integrated into higher-level constructs like agents and skills to create a more conversational experience.
voice-assistantAgent: A conversational agent persona that is defined to speak its responses aloud. It is instructed to create a concise spoken summary of its text response and use ElevenLabs TTS to voice it.voice-responsesSkill: This skill enables Claude to automatically speak responses when it detects voice-related requests (e.g., "speak," "read aloud") or for accessibility purposes. It provides guidelines for creating concise spoken summaries (max 50 words).
Sources: agents/voice-assistant.md, skills/voice-responses.md
Summary
The ElevenLabs API integration for Claude Code provides a powerful and flexible voice interface. Through a series of simple commands, it exposes advanced features of the ElevenLabs v3 API, including expressive text-to-speech with emotional tags, multi-speaker dialogue generation, custom voice design, and accurate audio transcription. Furthermore, its automated hook system enhances the user experience by providing timely, hands-free audio notifications for key events, making it a significant enhancement for accessibility and interactive coding sessions.
Sources: README.md, demo-script.md