AI Content Formatting

Relevant source files

The following files were used as context for generating this wiki page:

• apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt
• apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/YouTubeAiContentFormatter.kt
• apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/JsoupAiContentFormatter.kt
• apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/XApiAiContentFormatter.kt
• apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/FirecrawlAiContentFormatter.kt
• apps/web/src/features/sources/components/SourceContent.tsx
• apps/api/src/test/kotlin/com/briefy/api/application/source/SourceContentFormatterServiceTest.kt

AI Content Formatting is a critical post-processing stage in the Briefy pipeline that transforms raw extracted text from various sources—such as web scrapers, social media APIs, and video transcripts—into clean, readable Markdown. This system ensures that noise like menu items, sidebars, and metadata are removed, and that unstructured content like video captions are broken into logical paragraphs.

The formatting process is primarily orchestrated by the SourceContentFormatterService and executed by specialized implementation components that interact with Large Language Models (LLMs) via an AiAdapter.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt, apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/YouTubeAiContentFormatter.kt

System Architecture

The formatting system follows a provider-based architecture where specific formatters are selected based on the ExtractionProviderId.

Logic Flow and Components

The following diagram illustrates the high-level flow of a formatting request:

The system ensures that only "Active" sources are processed and prevents redundant AI calls if the content is already marked as aiFormatted.
Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt:33-85

Core Service Components

Component	Responsibility
SourceContentFormatterService	Orchestrates the formatting lifecycle, handles transactions, and updates snapshots.
ExtractionContentFormatter	Interface defining supports(extractorId) and format(content, provider, model).
AiAdapter	Infrastructure component that communicates with external LLM providers (e.g., Google GenAI, ZhipuAI).
UserAiSettingsService	Resolves the specific AI model and provider configured for the user's formatting use case.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt:20-28, apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/JsoupAiContentFormatter.kt:10-14

Specialized Formatters

Different extraction sources require unique prompts and processing rules to achieve optimal results.

Web Content (Jsoup & Firecrawl)

The JsoupAiContentFormatter and FirecrawlAiContentFormatter share similar logic aimed at stripping boilerplate.

• Rules: Remove menu items, sidebars, footers, and post metadata (authors, dates).
• Goal: Infer feasible markdown formatting while keeping the core content identical to the original text.

Sources: apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/JsoupAiContentFormatter.kt:17-26, apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/FirecrawlAiContentFormatter.kt:17-26

Video Content (YouTube)

The YouTubeAiContentFormatter handles raw caption tracks.

• Configuration: Unlike other formatters, YouTube formatting is hardcoded to use google_genai with gemini-2.5-flash-lite.
• Logic: Captions are only formatted if the transcriptSource is "captions". If the transcript is generated by other means (like Whisper), formatting is skipped as not required.
• Goal: Break continuous caption streams into readable paragraphs at natural topic changes.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt:190-194, apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/YouTubeAiContentFormatter.kt:17-25

Social Media (X/Twitter)

The XApiAiContentFormatter processes posts and threads.

• Rules: Keep content identical; do not remove text.
• Logic: Specifically targets breaking transcripts into paragraphs at natural speaker changes or topic shifts.

Sources: apps/api/src/main/kotlin/com/briefy/api/infrastructure/formatting/XApiAiContentFormatter.kt:17-25

State Management and Error Handling

The formatting state is tracked in the Metadata object of the source.

Formatting States

State	Description
PENDING	Default state for new extractions that require formatting.
SUCCEEDED	AI formatting completed successfully; content updated to Markdown.
FAILED	Process failed due to AI errors, resolution failures, or empty output.
NOT_REQUIRED	Formatting was skipped (e.g., unsupported extractor or already clean content).

Sources: apps/web/src/lib/api/types.ts:24-27, apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt:56-111

Failure Reasons

The service logs specific failure reasons when a transition to FAILED occurs:

• ai_settings_resolution_failed: Unable to determine which AI model to use.
• llm_error: The external AI provider threw an exception or timed out.
• empty_formatted_content: The AI returned a blank response.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt:204-207

Shared Source Snapshots

To optimize performance and reduce AI costs, formatted content is synced back to the SharedSourceSnapshot system.

Eligibility for Snapshot Update:

1. Source cannot be manually entered (EXTRACTION_PROVIDER_MANUAL).
2. The existing latest snapshot must be ACTIVE and have a matching extractionProvider.
3. The existing snapshot must not already be aiFormatted.

Sources: apps/api/src/main/kotlin/com/briefy/api/application/source/SourceContentFormatterService.kt:140-184

User Interface Interaction

The frontend monitors the formattingState to provide real-time feedback to the user.

• Pending Notice: Shown when the state is pending. It provides a "See raw content" button as a fallback while the AI works.
• Failed Notice: Displayed when state is failed. It presents the formattingFailureReason (formatted for readability) and offers a "Retry formatting" button.
• YouTube Specifics: The UI displays a dedicated "Transcript" header for video sources and can render embedded video players alongside the formatted text.

Sources: apps/web/src/features/sources/components/SourceContent.tsx:43-118

Conclusion

The AI Content Formatting system serves as the primary bridge between raw data extraction and high-quality user consumption. By leveraging specialized LLM prompts and a robust state management service, it ensures that every source in the knowledge graph is presented in a clean, professional Markdown format, regardless of its original origin.