Testing & Benchmarking
Relevant source files
The following files were used as context for generating this wiki page:
- benchmark/src/main/kotlin/dev/patrickgold/florisboard/benchmark/StartupBenchmark.kt
- benchmark/src/main/kotlin/dev/patrickgold/florisboard/benchmark/BaselineProfileGenerator.kt
- app/src/test/kotlin/dev/patrickgold/florisboard/ime/nlp/TypingSpeedMetricsTest.kt
- lib/snygg/src/test/kotlin/org/florisboard/lib/snygg/SnyggStylesheetEditorTest.kt
- lib/snygg/src/test/kotlin/org/florisboard/lib/snygg/SnyggStylesheetTest.kt
- app/src/main/kotlin/dev/patrickgold/florisboard/lib/devtools/Devtools.kt
- gradle/libs.versions.toml
The Ownkey Keyboard project employs a multi-layered approach to validation, encompassing unit testing, performance benchmarking, and real-time telemetry via developer tools. This infrastructure ensures that the keyboard remains responsive—particularly when integrating AI features—and maintains functional correctness across its various modules, including the Natural Language Processing (NLP) engine and the Snygg theme library.
Sources: AGENTS.md, CONTRIBUTING.md
Macrobenchmarking & Performance
The project uses the androidx.benchmark library to measure critical performance metrics, primarily focusing on application startup time. Benchmarks are categorized by the state of the application process (Cold, Warm, or Hot) and the level of Ahead-of-Time (AOT) compilation applied.
Startup Benchmarking
The AbstractStartupBenchmark class provides a standardized framework for measuring the time taken from activity launch until the first frame is rendered. It utilizes MacrobenchmarkRule to automate repeated measurements and system tracing.
This diagram illustrates the automated loop used to capture consistent startup measurements across multiple iterations.
Sources: benchmark/src/main/kotlin/dev/patrickgold/florisboard/benchmark/StartupBenchmark.kt:47-81
Compilation Modes
Benchmarks are executed under different CompilationMode settings to evaluate the impact of Baseline Profiles and full ART compilation:
| Mode | Description |
|---|---|
CompilationMode.None() | No pre-compilation; simulates first-launch performance. |
CompilationMode.Partial() | Uses Baseline Profiles to optimize critical code paths. |
CompilationMode.Full() | Full AOT compilation of the entire application. |
Sources: benchmark/src/main/kotlin/dev/patrickgold/florisboard/benchmark/StartupBenchmark.kt:56-70
Baseline Profile Generation
To optimize real-world performance, the project includes a BaselineProfileGenerator. This tool interacts with the UI (starting activities and waiting for idle states) to record a profile of the classes and methods used during critical user journeys.
Sources: benchmark/src/main/kotlin/dev/patrickgold/florisboard/benchmark/BaselineProfileGenerator.kt:31-41
Unit Testing & Functional Validation
Unit tests are written using the Kotest and Kotlin Test frameworks. They target logic-heavy components like the Snygg theme engine and the NLP metrics tracker.
Typing Metrics Validation
The TypingSpeedMetricsTest ensures accurate calculation of user performance data. It validates aggregates for suggestion latency, acceptance rates, and keystroke efficiency.
| Metric | Calculation Logic / Test Focus |
|---|---|
suggestionLatencyP95Ms | 95th percentile of recorded suggestion delivery times. |
top3AcceptanceRate | Ratio of accepted suggestions within the first three candidates. |
falseAutocorrectRatio | Ratio of autocorrect applications that were subsequently undone by the user. |
keystrokesPerWord | Total text input keystrokes divided by committed word count. |
Sources: app/src/test/kotlin/dev/patrickgold/florisboard/ime/nlp/TypingSpeedMetricsTest.kt:31-60
Theme Engine (Snygg) Testing
The Snygg library tests focus on JSON serialization/deserialization and the dynamic building of stylesheets. Tests cover:
- Schema Validation: Ensuring stylesheets include mandatory version markers.
- Lenient Parsing: Verifying the engine can recover from missing properties or invalid rules when configured.
- Variable References: Validating that
@definesrules correctly map variables (e.g.,--primary-color) to properties.
Sources: lib/snygg/src/test/kotlin/org/florisboard/lib/snygg/SnyggStylesheetTest.kt, lib/snygg/src/test/kotlin/org/florisboard/lib/snygg/SnyggStylesheetEditorTest.kt
Real-time Metrics & Debugging
The Devtools utility acts as an internal monitoring system, providing snapshots of the keyboard's state that can be used for manual benchmarking and troubleshooting.
Telemetry Generation Flow
The system can generate comprehensive logs for GitHub issue reports, combining system information with live application performance data.
This sequence shows how Devtools aggregates data from various subsystems to create a diagnostic report.
Sources: app/src/main/kotlin/dev/patrickgold/florisboard/lib/devtools/Devtools.kt:41-86
Key Metrics Tracked
Devtools captures several metrics that serve as ad-hoc benchmarks during development:
- Memory Usage: Java heap and Native heap usage (formatted via
UnitUtils). - Typing Efficiency: Keystrokes per word and autocorrect undo frequency.
- Suggestion Latency: P95 latency in milliseconds.
Sources: app/src/main/kotlin/dev/patrickgold/florisboard/lib/devtools/Devtools.kt:158-175, 219-255
Test Infrastructure Configuration
The project manages its testing dependencies through a centralized libs.versions.toml file.
| Dependency | Purpose | Version |
|---|---|---|
androidx-benchmark-macro | Macrobenchmark support for startup and UI testing. | 1.4.1 |
androidx-test-uiautomator | Automating device interactions during benchmarks. | 2.3.0 |
kotest | Primary functional testing framework. | 6.1.3 |
kotlinx-kover | Kotlin code coverage tool. | 0.9.7 |
Sources: gradle/libs.versions.toml:34-40, 93-102
Summary
Ownkey Keyboard prioritizes performance through rigorous startup benchmarking and ART optimization via Baseline Profiles. Functional integrity is maintained by targeted unit tests for NLP and theme modules, while the Devtools utility provides a bridge between automated testing and manual diagnostic capabilities, ensuring AI integrations do not compromise keyboard responsiveness.