Test Suite & Warning Config
Relevant source files
The following files were used as context for generating this wiki page:
The Jazz project utilizes a multi-layered testing and configuration system focused primarily on the jazz-next compiler target. The system ensures language consistency through automated policy checks, detailed compiler warning configurations, and a comprehensive suite of unit and integration tests. This infrastructure governs how the compiler interprets warning flags, enforces repository policies, and validates semantic correctness across various implementation slices.
Sources: jazz-next/README.md:1-25, scripts/check-spec-authority.sh:1-10
Warning Configuration System
The warning configuration in Jazz is managed through a specific precedence hierarchy and specialized modules within the jazz-next compiler. It supports standard -W token parsing and allows for granular control over diagnostic behavior.
Configuration Hierarchy
Warning settings are resolved using a specific merge precedence where the Command Line Interface (CLI) takes highest priority. The order of precedence is:
- CLI Flags: Direct arguments provided to the compiler executable.
- Environment Variables: System-level variables.
- Config Files: Persistent project or user configuration files.
- Defaults: Hardcoded baseline settings.
Sources: jazz-next/README.md:12-14
Architecture of Diagnostics
The warning system is decomposed into several functional components within the Haskell source tree:
| Component | File Path | Description |
|---|---|---|
| Warning Definitions | Warnings.hs | Defines warning categories, severity levels, and category parsing logic. |
| Config Resolution | WarningConfig.hs | Implements token parsing for -W flags and the precedence merge logic. |
| Diagnostic Payloads | Diagnostics.hs | Defines the shape of warning payloads and handles deterministic sorting of output. |
| Driver Integration | Driver.hs | Implements "warning-as-error" gating and controls semantic error propagation. |
Sources: jazz-next/README.md:12-21
The following diagram illustrates the flow of warning configuration from input to the compiler driver:
The diagram shows how various configuration sources are merged before being parsed and applied to the compiler's diagnostic engine.
Sources: jazz-next/README.md:12-23
Test Suite Architecture
The Jazz test suite is categorized by the compiler phase it targets, ranging from low-level lexing to high-level module resolution and runtime semantics.
Test Layout and Entrypoints
The primary entrypoint for running tests in the active compiler is the test-warning-config.sh script. This script executes a series of Haskell specifications using a runghc wrapper.
- Parser Tests: Validate the surface syntax, including lambda expressions, if-expressions, and ADT patterns.
- Semantic Tests: Check rebinding warnings (
W0001), type inference for if-expressions, and purity semantics. - Module Tests: Verify prelude loading, module resolution, and the module loader.
- CLI Tests: Ensure correct flag resolution and environmental variable handling.
Sources: jazz-next/README.md:27-35, jazz-next/scripts/test-warning-config.sh:13-39
Automated Policy Enforcement
Beyond unit tests, the project employs shell-based "checkers" to enforce architectural and documentation standards.
- Spec Authority Check: Ensures that documentation correctly identifies
jazz2as non-normative and anchors authority todocs/specandjazz-nexttests. - Docs Status Check: Verifies that the
README.mdanddocs/feature-status.mdare synchronized and meet maintenance checklist requirements. - Clarification Specs: Validates that specific specification files contain required sections like "Typing Rules" or "Canonical Desugaring".
Sources: scripts/check-spec-authority.sh:25-45, scripts/check-docs.sh:31-45, scripts/check-clarification-specs.sh:26-45
This sequence shows the execution flow for a single test specification within the test-warning-config.sh loop.
Sources: jazz-next/scripts/test-warning-config.sh:1-55
Repository Governance Policies
The test suite enforces a strict "Spec Authority" policy. This policy dictates that implementation work must only occur in the jazz-next/ directory, while jazz-hs/ and jazz2/ are treated as read-only historical or experimental references.
Policy Enforcement Rules
The check-spec-authority.sh script uses regex patterns to ensure documentation does not make unsupported normative claims about legacy paths.
| Policy Requirement | Regex Pattern Intent |
|---|---|
| Reference Only | Matches "jazz2" associated with "non-normative" or "reference-only". |
| Authority Anchor | Ensures authority is linked to docs/spec/ or jazz-hs behavior/tests. |
| Change Control | Validates that semantic changes require an RFC or decision record. |
Sources: scripts/check-spec-authority.sh:25-30, README.md:40-50
Summary
The Jazz Test Suite & Warning Config system provides a robust framework for maintaining the integrity of the jazz-next compiler. By combining a hierarchical warning configuration with exhaustive phase-based testing and automated policy enforcement, the project ensures that new implementations adhere to the normative language specification while preventing regressions in both code behavior and repository governance.
Sources: jazz-next/README.md:1-10, scripts/check-spec-authority.sh:60-65