AI Context Formatter
Relevant source files
The following files were used as context for generating this wiki page:
The AI Context Formatter is a specialized subsystem within MacParakeet designed to prepare and transform raw transcriptions into polished, high-quality text using Large Language Models (LLMs). Its primary purpose is to provide contextually aware text refinement, such as fixing disfluencies, grammar, and formatting, while ensuring the output remains faithful to the original recording.
This system acts as an optional polish layer on top of the deterministic Text Processing Pipeline. It leverages a combination of structured prompt templates, context window management, and safety validators to prevent common LLM failure modes like hallucinations or repetition loops.
Sources: spec/12-processing-layer.md, plans/active/2026-06-21-refinement-validator.md
System Architecture
The AI Context Formatter operates at the intersection of the text processing pipeline and the LLM service layer. It follows a structured flow where input text is formatted with metadata, processed by a provider (such as Anthropic, OpenAI, or a local MLX model), and then validated for quality.
Core Components
| Component | Description |
|---|---|
AIFormatter | Manages the primary logic for building system prompts and user prompts for LLM refinement. |
TranscriptAIContextFormatter | A utility for injecting rich metadata (timestamps, speaker labels, file info) into the LLM context. |
RefinementValidator | A deterministic safety gate that judges if the LLM output is a safe replacement for the baseline. |
LLMService | The abstraction layer that routes requests to various cloud or local providers. |
Sources: Sources/MacParakeetCore/TextProcessing/AIFormatter.swift, Sources/MacParakeetCore/TextProcessing/RefinementValidator.swift
Data Flow Diagram
The following diagram illustrates how a raw transcript moves through the AI Context Formatter to become a refined output.
The system ensures that if the LLM fails or produces low-quality results, the user always falls back to the reliable deterministic baseline.
Sources: plans/active/2026-06-21-refinement-validator.md
Context Construction Logic
The AIFormatter is responsible for assembling the prompt sent to the LLM. It uses a specific instruction set that guides the model to perform "light cleanup" rather than creative rewriting.
System Prompt Requirements
The formatter instructs the model to adhere to several strict rules:
- Faithfulness: Do not add information not present in the original.
- Grammar & Punctuation: Fix disfluencies (um, uh) and apply standard casing.
- ITN (Inverse Text Normalization): Properly format numbers, dates, and technical terms.
- Structural Preservation: Maintain the core meaning while improving readability.
Sources: Sources/MacParakeetCore/TextProcessing/AIFormatter.swift:20-45, plans/active/2026-06-27-on-device-local-llm.md
Metadata Injection
The TranscriptAIContextFormatter enriches the prompt by adding relevant metadata. This helps the LLM understand the context of the recording, such as whether it originated from a specific application or a meeting.
// Example of context metadata assembly
let context = """
Source: \(metadata.sourceFilename)
Date: \(metadata.creationDate)
Duration: \(metadata.duration)
App: \(metadata.activeAppID ?? "Unknown")
"""
Sources: Sources/MacParakeetCore/TextProcessing/TranscriptAIContextFormatter.swift:10-25
Refinement Validation
To protect against LLM "hallucinations" or "runaway" output, the system includes a RefinementValidator. This component runs synchronous, deterministic checks on the LLM's response before it is saved or pasted.
Validation Criteria
| Reason | Logic |
|---|---|
lengthRunaway | Rejects if the output is implausibly longer than the input (e.g., model adds commentary). |
repetitionLoop | Detects back-to-back n-gram repeats (e.g., "the team the team the team"). |
lowContentOverlap | Ensures a sufficient fraction of "content words" from the original survived. |
empty | Rejects if the model returns whitespace or blank text. |
Sources: Sources/MacParakeetCore/TextProcessing/RefinementValidator.swift:15-40, plans/active/2026-06-21-refinement-validator.md
Validation Logic Flow
Performance and Constraints
The system manages different "Lanes" for refinement, each with its own limits on context size and validation strictness.
| Lane | Input Cap (Chars) | Growth Limit | Min Overlap |
|---|---|---|---|
Dictation | ~12,000 | 1.8x | 0.5 |
Transcription | ~60,000 | 2.0x | 0.5 |
Sources: Sources/MacParakeetCore/TextProcessing/AIFormatter.swift:12, Sources/MacParakeetCore/TextProcessing/RefinementValidator.swift:45-55
Context Window Management
For very long transcripts, the system implements middle-truncation (keeping the first 45% and last 45%) to ensure the LLM receives the most relevant context without exceeding its token limits.
Sources: plans/completed/2026-02-llm-provider-integration.md
Summary
The AI Context Formatter provides a high-fidelity refinement layer for MacParakeet's transcriptions. By combining rich context injection with deterministic validation, it ensures that AI-enhanced text remains accurate and faithful to the user's original speech. It is designed to be provider-agnostic, supporting both cloud APIs and local MLX models, while prioritizing safety through silent fallbacks to the deterministic baseline in the event of LLM failure.