Text Processing Pipeline
Relevant source files
The following files were used as context for generating this wiki page:
- Sources/MacParakeetCore/TextProcessing/TextProcessingPipeline.swift
- spec/adr/004-deterministic-pipeline.md
- Sources/MacParakeetCore/TextProcessing/README.md
- spec/07-text-processing.md
- Tests/MacParakeetTests/TextProcessing/TextProcessingPipelineTests.swift
- plans/completed/2026-02-v0.2-clean-pipeline.md
The Text Processing Pipeline is a deterministic system designed to transform raw Speech-to-Text (STT) output into polished, "clean" text. Unlike LLM-based refinement, which can be unpredictable and introduce latency, this pipeline is a pure function that operates in sub-millisecond time, ensuring that the same input always produces the same output and optional post-paste actions.
The pipeline is primarily utilized in the project's "Clean" processing mode. It addresses common STT artifacts such as filler words ("um", "uh"), casing errors, and domain-specific vocabulary while providing user-facing features like text snippets and voice-triggered actions (e.g., "Voice Return").
Sources: spec/07-text-processing.md:5-10, spec/adr/004-deterministic-pipeline.md:14-25, Sources/MacParakeetCore/TextProcessing/README.md:10-15
Architecture and Core Principles
The pipeline is implemented as a pure value type (TextProcessingPipeline) with no side effects, I/O, or internal state. This architecture allows for exhaustive testing and high-performance execution, typically under 5ms, which is critical for maintaining real-time dictation flow.
Pipeline Logic Flow
The following diagram illustrates the sequential stages of the text processing engine:
The order of these steps is considered "load-bearing." For instance, filler removal happens before custom word replacement to prevent custom rules from matching substrings within filler words. Similarly, action extraction occurs before snippet expansion so that expansion rules do not mangle action trigger phrases.
Sources: Sources/MacParakeetCore/TextProcessing/README.md:37-56, spec/adr/004-deterministic-pipeline.md:73-81
Detailed Processing Steps
1. Filler Removal
This stage strips "always-safe" hesitation sounds that do not carry semantic meaning. The list is intentionally narrow to avoid deleting words like "like" or "right" which can be context-dependent.
- Target Words: "um", "uh", "umm", "uhh".
- Implementation: Uses pre-compiled
NSRegularExpressionwith word boundaries (\b) to ensure partial matches (e.g., "umbrella") are preserved.
Sources: Sources/MacParakeetCore/TextProcessing/TextProcessingPipeline.swift:63-71, spec/07-text-processing.md:16-24
2. Custom Word Replacements
Users can define vocabulary anchors and corrections. This step enforces correct casing for technical terms (e.g., "kubernetes" → "Kubernetes") or fixes phonetic errors (e.g., "aye pee eye" → "API").
- Matching: Case-insensitive with whole-word boundaries.
- Reuse: Meeting transcripts reuse this specific step (via
CustomWordReplacer) to fix verbatim records without applying dictation-only styling.
Sources: spec/07-text-processing.md:26-36, Sources/MacParakeetCore/TextProcessing/README.md:25-30
3. Trailing Action Extraction
If a dictation ends with a specific trigger phrase associated with a KeyAction (like returnKey), the pipeline extracts that action and removes the trigger from the text. This powers "Voice Return" functionality.
- Punctuation Tolerance: Matches triggers even if followed by terminal punctuation (e.g., "press return.").
- Execution: Happens before snippet expansion to prevent trigger mangling.
Sources: Sources/MacParakeetCore/TextProcessing/TextProcessingPipeline.swift:101-118, spec/07-text-processing.md:46-52
4. Snippet Expansion
Natural language trigger phrases are replaced with their full expansion text.
- Sorting: Triggers are sorted by length in descending order (longest-first) to prevent a shorter trigger from partially matching a longer one.
- Example: "my signature" → "Best regards, David".
Sources: Sources/MacParakeetCore/TextProcessing/TextProcessingPipeline.swift:135-145, spec/07-text-processing.md:54-60
5. Whitespace Cleanup and Insertion Styling
The final stage normalizes the text and applies the selected DictationInsertionStyle.
| Style | Description |
|---|---|
| Sentence | Default style. Capitalizes the first letter and preserves terminal punctuation. |
| Inline | Removes terminal punctuation and lowercases the leading letter (preserving acronyms/camelCase) for inserting into existing text. |
The cleanup also collapses multiple spaces and ensures punctuation is correctly spaced (e.g., removing the space before a period).
Sources: Sources/MacParakeetCore/TextProcessing/TextProcessingPipeline.swift:167-185, spec/07-text-processing.md:65-75
Data Structures
The pipeline interacts with several core models and produces a TextProcessingResult.
TextProcessingResult
This structure carries the transformed text and metadata about the processing session.
public struct TextProcessingResult: Sendable {
public let text: String
public let expandedSnippetIDs: Set<UUID>
public let postPasteAction: KeyAction?
}
Sources: Sources/MacParakeetCore/TextProcessing/TextProcessingPipeline.swift:54-58, Sources/MacParakeetCore/TextProcessing/README.md:18-21
Custom Word Model
| Field | Type | Description |
|---|---|---|
word | TEXT | The phrase to match (case-insensitive). |
replacement | TEXT | The corrected phrase (null if vocabulary anchor). |
isEnabled | BOOLEAN | Toggle for active processing. |
Sources: spec/07-text-processing.md:88-96
Summary of Processing Modes
The system provides two distinct paths for text delivery based on user preference.
| Mode | Pipeline Complexity | Latency | Use Case |
|---|---|---|---|
| Raw | None | 0ms | Debugging or verbatim requirements. |
| Clean | 5-Step Deterministic | <1ms | General dictation and polished output. |
Sources: spec/07-text-processing.md:78-84, spec/adr/004-deterministic-pipeline.md:43-50
The Text Processing Pipeline serves as a high-performance alternative to LLM refinement, prioritizing predictability and low latency. By chaining specific, rule-based transformations, it allows MacParakeet to deliver clean, professionally formatted text instantly while maintaining user-defined customizations through snippets and vocabulary anchors.