Testing Strategy
Relevant source files
The following files were used as context for generating this wiki page:
PRKS employs a multi-layered testing strategy to ensure the correctness, performance, and visual integrity of the research library. The system prioritizes fast-executing unit and API tests while reserving heavy browser-based End-to-End (E2E) tests for final verification of vertical slices. This strategy is strictly governed by scoped agent instructions that ensure tests target the correct production domains and maintain strict data isolation.
Sources: AGENTS.md:120-130, tests/AGENTS.md:1-15
Test Hierarchy and Classification
PRKS categorizes tests by their scope and execution environment. The project favors a "fail-fast" approach, encouraging developers to use unit tests and Node selftests before running the full browser suite.
| Test Type | Scope | Execution Command |
|---|---|---|
| Unit Tests | Python standard library and domain logic. | python run_tests.py |
| Node Selftests | Browser-side logic (e.g., Markdown sanitization). | tests/browser/*.js |
| API Tests | HTTP adapter and domain controller contracts. | python run_tests.py |
| E2E Tests | Browser workflows using Playwright/Chromium. | scripts/e2e agent or python run_tests.py --e2e |
| UX Tour | Human-oriented interaction review with artifacts. | python run_tests.py --ux-tour |
| Design System | Visual contract, token, and accessibility guards. | python tests/test_design_system.py |
Sources: AGENTS.md:104-108, tests/ux_tour/COVERAGE.md:5-10, tests/test_design_system.py:1-5
Test Execution Flow
The following diagram illustrates the recommended progression of testing during the development lifecycle.
The diagram shows the transition from fast, low-level tests to heavy, artifact-producing integration tests.
Sources: AGENTS.md:112-117, tests/ux_tour/AGENTS.md:5-15
Browser E2E Policy
E2E tests use Playwright to exercise the full application stack in Chromium. The project enforces a strict "never run while iterating" policy for the full E2E suite to prevent slow development loops.
Sources: AGENTS.md:112-117
- Isolation: Tests must never target the production
data/directory. They usePRKS_TESTING=1to redirect storage todata_testing/. - Parallelization: The full E2E gate runs with 4 workers by default, but agents are instructed to use 1 worker when debugging specific failures.
- Timeouts: The suite enforces a 1200-second hard limit (
PRKS_E2E_FULL_TIMEOUT). - Agent Loop: Developers should prefer
scripts/e2e agent, which only runs affected tests and disables pointer capture for speed.
Sources: AGENTS.md:108-115, AGENTS.md:129-132
UX Interaction Tour
The UX Interaction Tour is an opt-in suite that produces reviewable artifacts (videos, traces, and screenshots). It is designed to evaluate human-oriented capabilities that automated E2E tests might overlook.
Sources: tests/ux_tour/AGENTS.md:5-15
- Interaction Method: Tours must use real user-visible controls (buttons, menus, command palette) rather than internal JavaScript calls like
window.prksNavigate. - Artifacts: Passing scenarios delete heavy artifacts (video/trace) unless
PRKS_UX_RECORD=1is set. - Coverage: The tour covers high-level surfaces such as Shell navigation, Workspace tiling, and managed PDF annotation.
Sources: tests/ux_tour/AGENTS.md:10-25, tests/ux_tour/COVERAGE.md:15-30
Performance and Baseline Testing
PRKS uses specialized scripts to record browser and resource baselines to prevent performance regressions during architectural transitions.
Sources: scripts/b5_browser_baselines.py:1-15
Sequence of capturing performance baselines for the Vue cutover.
Sources: scripts/b5_browser_baselines.py:100-150
Design System and Accessibility Guards
The tests/test_design_system.py module enforces visual and accessibility contracts. It validates that the implementation matches the DESIGN.md authority.
Sources: tests/test_design_system.py:1-20
- Token Guards: Verifies the existence and values of semantic tokens (e.g.,
--bg-color,--accent). - Accessibility: Ensures every visible input has an accessible name (labels or ARIA attributes).
- CSS Policy: Prevents the use of "dead code" (e.g., writing a border longhand before a shorthand) and enforces the quiet selected state (surface changes rather than loud purple outlines).
Sources: tests/test_design_system.py:65-85, tests/test_design_system.py:220-240
Data and Storage Safety
The testing infrastructure strictly isolates test data to prevent accidental mutation of production libraries.
| Environment Variable | Purpose in Testing |
|---|---|
PRKS_TESTING | Set to 1 to enable testing mode and stricter path checks. |
PRKS_STORAGE | Defaults to data_testing/; refuses data/ or root /data. |
PRKS_FOR_PROCESSING_DIR | Explicitly cleared during test runs. |
PRKS_LOG_FILE | Redirected to a testing-specific log path. |
Sources: AGENTS.md:129-135, scripts/b5_browser_baselines.py:730-745
The PRKS testing strategy ensures that technical correctness, user experience, and performance are validated through a tiered approach, starting from the smallest code units up to human-oriented interaction tours. Final validation always requires the full parallelized E2E gate.