Audio Narration System

Relevant source files

The following files were used as context for generating this wiki page:

• apps/api/src/main/kotlin/com/briefy/api/application/source/SourceNarrationService.kt
• apps/api/src/main/kotlin/com/briefy/api/application/source/SourceDto.kt
• apps/api/src/test/kotlin/com/briefy/api/domain/knowledgegraph/source/SourceTest.kt
• apps/api/src/test/kotlin/com/briefy/api/api/SourceNarrationControllerTest.kt
• apps/api/src/test/kotlin/com/briefy/api/application/source/SourceNarrationServiceTest.kt
• apps/web/src/lib/api/sources.ts
• apps/web/src/lib/api/types.ts

The Audio Narration System is a specialized module designed to convert text-based source content into high-quality audio summaries or narrations using Text-to-Speech (TTS) technology. It primarily leverages the ElevenLabs API for synthesis and incorporates a sophisticated caching mechanism to optimize performance and reduce costs by reusing previously generated audio for identical content.

The system manages the entire lifecycle of a narration request, from initial estimation and user request through asynchronous processing, cloud storage, and delivery via pre-signed URLs. It is integrated into the "Source" domain, allowing users to listen to summarized content directly within the application.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceNarrationService.kt, apps/api/src/main/kotlin/com/briefy/api/application/source/SourceDto.kt

System Architecture and Components

The system is built as a service-oriented architecture within the backend API, utilizing a combination of domain entities, application services, and infrastructure adapters.

Key Components

Component	Description
SourceNarrationService	The primary application service orchestrating narration requests, processing, and audio refreshing.
ElevenLabsTtsAdapter	Infrastructure adapter responsible for communicating with the ElevenLabs TTS API.
AudioStorageService	Handles the uploading of MP3 files to cloud storage and generation of pre-signed URLs.
SharedAudioCache	A domain entity and repository used to store and retrieve metadata about synthesized audio based on content hashes.
MarkdownStripper	A utility that removes Markdown syntax from source text to provide clean plaintext for TTS synthesis.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceNarrationService.kt:25-45, apps/api/src/test/kotlin/com/briefy/api/application/source/SourceNarrationServiceTest.kt:35-50

Data Flow for Narration Requests

The following diagram illustrates the sequence of events from a user requesting narration to the delivery of the audio content.

Narration Lifecycle and States

The system tracks the progress of narration through specific states stored within the Source entity.

Narration States

State	Description
NOT_GENERATED	Default state; no narration has been requested yet.
PENDING	Narration is currently being processed or is in the queue.
SUCCEEDED	Audio has been successfully generated and is available for playback.
FAILED	An error occurred during synthesis or storage.

Sources: apps/web/src/lib/api/types.ts:50, apps/api/src/test/kotlin/com/briefy/api/domain/knowledgegraph/source/SourceTest.kt:145-170

Failure Handling and Retries

When a narration fails, the system records a narrationFailureReason. The NarrationFailureCatalog maps these internal codes to user-friendly messages and determines if the failure is retryable.

• Non-Retryable Errors: paid_plan_required, invalid_api_key, quota_exceeded, content_too_long, empty_plaintext_content.
• Retryable Errors: too_many_concurrent_requests, system_busy, audio_storage_failed, tts_generation_failed.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceDto.kt:116-146, apps/api/src/main/kotlin/com/briefy/api/application/source/SourceNarrationService.kt:70-90

Technical Implementation Details

Content Hashing and Deduplication

To avoid redundant API calls to ElevenLabs, the system uses NarrationContentHashing. Before synthesis, the source text is stripped of Markdown and hashed. The SharedAudioCacheRepository is queried using the contentHash, voiceId, and modelId.

val cachedAudio = transactionTemplate.execute {
    NarrationContentHashing.lookupHashes(loaded.contentText)
        .firstNotNullOfOrNull { hash ->
            sharedAudioCacheRepository.findByContentHashAndVoiceIdAndModelId(
                hash,
                ttsProperties.voiceId,
                ttsProperties.modelId
            )
        }
}

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceNarrationService.kt:127-135, apps/api/src/test/kotlin/com/briefy/api/application/source/SourceNarrationServiceTest.kt:105-125

Audio Storage and URL Refreshing

Synthesized audio is stored in MP3 format. Because pre-signed URLs provided by cloud storage providers often expire, the system includes a refreshAudio function. This endpoint generates a fresh pre-signed URL for existing audio content without re-synthesizing the text.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceNarrationService.kt:92-114, apps/api/src/test/kotlin/com/briefy/api/api/SourceNarrationControllerTest.kt:125-145

Frontend Integration

The frontend interacts with the narration system through several API endpoints defined in the web client.

export async function narrateSource(id: string): Promise<Source> {
  return apiPost<Source>(`/api/sources/${id}/narrate`)
}

export async function getNarrationEstimate(id: string): Promise<NarrationEstimateResponse> {
  return apiGet<NarrationEstimateResponse>(`/api/sources/${id}/narrate/estimate`)
}

Sources: apps/web/src/lib/api/sources.ts:101-120, apps/web/src/lib/api/types.ts:250-265

API Endpoints

Method	Endpoint	Description
POST	/api/sources/{id}/narrate	Initiates a new narration request.
POST	/api/sources/{id}/narrate/retry	Retries a previously failed narration.
GET	/api/sources/{id}/narrate/estimate	Returns the estimated character count and model to be used.
GET	/api/sources/{id}/audio	Retrieves the audio metadata and a fresh pre-signed URL.

Sources: apps/api/src/test/kotlin/com/briefy/api/api/SourceNarrationControllerTest.kt:55-120, apps/web/src/lib/api/sources.ts:101-125

The Audio Narration System ensures a cost-effective and responsive user experience by combining high-quality TTS synthesis with intelligent content caching and asynchronous processing.