Latin NLP & Autocorrect
Relevant source files
The following files were used as context for generating this wiki page:
- app/src/main/kotlin/dev/patrickgold/florisboard/ime/nlp/latin/LatinLanguageProvider.kt
- app/src/main/kotlin/dev/patrickgold/florisboard/ime/nlp/NlpManager.kt
- app/src/test/kotlin/dev/patrickgold/florisboard/ime/nlp/latin/LatinPredictionShortcutsTest.kt
- app/src/test/kotlin/dev/patrickgold/florisboard/ime/nlp/latin/HighCertaintyAutocorrectPolicyTest.kt
- app/src/test/kotlin/dev/patrickgold/florisboard/ime/nlp/latin/AppSpecificAutocorrectProfilePolicyTest.kt
- app/src/main/res/values/strings.xml
The Latin NLP (Natural Language Processing) and Autocorrect system in Ownkey Keyboard provides intelligent text prediction, spelling correction, and automatic error remediation for Latin-script languages such as English and Dutch. This system is designed to be privacy-first, performing all linguistic analysis locally on the device without transmitting keystrokes to external servers.
At its core, the system utilizes frequency-weighted dictionaries, Damerau-Levenshtein distance algorithms for typo detection, and context-aware language weights to rank candidates. It also incorporates a "High Certainty" policy to ensure that autocorrect only triggers when there is a significant statistical lead over alternative candidates.
Sources: LatinLanguageProvider.kt:40-60, README.md:43-48
System Architecture
The NLP system is orchestrated by the NlpManager, which serves as the central hub for various suggestion providers (Latin, Han, Emoji, and Clipboard). The LatinLanguageProvider specifically implements the SpellingProvider and SuggestionProvider interfaces to handle Latin scripts.
Core Components
| Component | Description |
|---|---|
NlpManager | Manages active candidates and coordinates between different provider types. |
LatinLanguageProvider | Primary implementation for Latin NLP; handles dictionary loading, candidate ranking, and spelling logic. |
LanguageModel | Stores word frequencies, a "delete index" for fuzzy matching, and prediction shortcuts. |
MixedLanguageScoringPolicy | Blends confidence scores when multiple languages are active in a subtype. |
HighCertaintyAutocorrectPolicy | Evaluates if a candidate is statistically significant enough to be auto-committed. |
Sources: NlpManager.kt:45-75, LatinLanguageProvider.kt:100-140
Data Flow for Suggestions
When the user types, the system follows a specific sequence to generate and rank candidates:
The diagram shows the logic flow from raw user input to the final delivery of ranked suggestion candidates. Sources: LatinLanguageProvider.kt:211-300
Dictionary & Model Management
The system uses frequency-based dictionaries to prioritize common words. Dictionaries for English (en_50k.txt) and Dutch (nl_50k.txt) are included as assets.
Fuzzy Matching with Delete Index
To handle typos efficiently, the system builds a "delete index" during the model loading phase. This index maps words with one character removed to their original forms, allowing for fast $O(1)$ lookups of potential corrections within an edit distance of 1.
- Max Edit Distance: 1 (default for autocorrect).
- Max Delete Word Count: 20,000 words.
- Min Word Length for Typo Correction: 4 characters.
Sources: LatinLanguageProvider.kt:49-70, LatinLanguageProvider.kt:465-515
Candidate Ranking Logic
The rankSuggestionCandidate function calculates a score based on several factors:
- Frequency Score: Ratio of word frequency to the maximum frequency in the model.
- Prefix Boost: Adds weight (approx. 0.35) if the candidate is a prefix match.
- Distance Penalty: Reduces score based on Damerau-Levenshtein distance (0, 1, or >1).
- Length Penalty: Penalizes large differences in length between input and candidate.
Sources: LatinLanguageProvider.kt:660-675
Autocorrect Policies
Ownkey Keyboard employs a multi-layered policy to prevent "ducking" errors (unwanted corrections).
High Certainty Autocorrect
This policy evaluates candidates before they are eligible for "Auto-Commit." A candidate must meet specific thresholds:
- Min Confidence: Usually between 50% and 99% (configurable).
- Confidence Gap: The top candidate must lead the runner-up by a defined margin (e.g., 12%).
- Input Length: Autocorrect is typically restricted for very short tokens (e.g., < 3 characters).
Sources: HighCertaintyAutocorrectPolicyTest.kt:25-45, LatinLanguageProvider.kt:718-735
App-Specific Aggressiveness
The system can adjust its autocorrect behavior based on the active application.
This sequence illustrates how the system dynamically adjusts sensitivity based on the app context. Sources: AppSpecificAutocorrectProfilePolicyTest.kt:35-55, LatinLanguageProvider.kt:737-750
Predictive Features
Next-Word Prediction
If the input is empty and the cursor is at a word boundary, the system attempts to predict the next word. It extracts recent context tokens (window size of 6) and evaluates bigram probabilities across active language models.
Rapid Vocabulary Learning
The RapidPersonalVocabularyLearner tracks accepted and reverted suggestions. If a user consistently accepts a word not in the main dictionary, it is promoted to the UserDictionary with a frequency boost. Conversely, reverting an autocorrected word can trigger "Never Correct" logic for that token.
Sources: LatinLanguageProvider.kt:320-350, LatinLanguageProvider.kt:550-600
Configuration Options
NLP behavior is controlled via several preferences defined in the system.
| Preference Key | Type | Description |
|---|---|---|
suggestion.enabled | Boolean | Toggles the display of the suggestion bar. |
correction.highCertaintyAutocorrectEnabled | Boolean | Enables the high-certainty logic for auto-commit. |
correction.appSpecificAutocorrectProfilesEnabled | Boolean | Toggles context-aware aggressiveness tuning. |
suggestion.blockPossiblyOffensive | Boolean | Filters candidates against an internal offensive word list. |
Sources: strings.xml:350-380, NlpManager.kt:130-150
Conclusion
The Latin NLP system provides a balance between typing speed and accuracy by combining traditional dictionary lookups with modern fuzzy matching and context-aware policies. By utilizing local frequency models and a "High Certainty" commitment logic, it aims to provide helpful suggestions while minimizing intrusive or incorrect automated edits.
Sources: LatinLanguageProvider.kt:800-820, VOXTRAL_KEYBOARD_PLAN.md:25-30