Validation Engine

Relevant source files

The following files were used as context for generating this wiki page:

• scripts/validate.py
• scripts/validate_candidate.py
• scripts/test_validate_candidate.py
• scripts/preflight.py
• scripts/test_workflow_safety.py
• README.md
• REVIEW.md
• AGENTS.md

The Validation Engine is a multi-stage verification system designed to ensure the integrity, security, and schema-compliance of the Numan official registry. It operates as a data-only repository guardrail, mirroring the verification logic found in the production Numan client to prevent trust bypasses, private-key leakage, and index corruption.

The engine encompasses static analysis, schema validation, cryptographic signature verification, and automated lifecycle proofs. It serves as the primary gatekeeper for CI/CD workflows, ensuring that only verified artifacts and signed manifests reach the production GitHub Pages environment.

Sources: README.md:99-105, scripts/validate.py:9-15, AGENTS.md:7-10

Architecture and Core Components

The validation architecture is structured into multiple discrete steps, ranging from local pre-flight checks to production-level signature verification.

Validation Pipeline Flow

The following diagram illustrates the sequence of checks performed when validating a candidate package or the full index:

The pipeline ensures that basic safety and formatting are correct before proceeding to heavy operations like network-based artifact downloads or cryptographic verification.
Sources: scripts/validate_candidate.py:20-28, REVIEW.md:7-18, scripts/validate.py:214-255

Component Summary

Component	Responsibility	Relevant Files
Secret Scanner	Detects private key material (*.key, *.pem) in git-tracked files.	scripts/scan_for_secrets.py, SECURITY.md
Preflight	Checks key/workflow consistency without network access.	scripts/preflight.py, README.md
Index Validator	Handles schema validation and Ed25519 signature verification.	scripts/validate.py
Candidate Harness	Orchestrates Stage 5 validation for new package specs.	scripts/validate_candidate.py
Parser Check	Ensures index compatibility with the Numan Rust parser.	tools/numan-parser-check/

Sources: README.md:46-60, AGENTS.md:20-30, REVIEW.md:31-35

Core Validation Logic

Canonical JSON Serialization

A critical invariant of the engine is the use of Canonical JSON. The signature is calculated over the canonical bytes, not the raw file contents. This involves sorting object keys, using compact encoding, and removing whitespace.

def canonical_json(value):
    if isinstance(value, dict):
        items = sorted((canonical_json(k), canonical_json(v)) for k, v in value.items())
        return "{" + ",".join(f"{k}:{v}" for k, v in items) + "}"
    if isinstance(value, list):
        return "[" + ",".join(canonical_json(v) for v in value) + "]"
    # ... handles strings, bools, and null

Sources: scripts/validate.py:21-35, README.md:82-88

Cryptographic Verification

The engine verifies detached Ed25519 signatures stored in index.json.sig. It uses the cryptography library to ensure that the detached signature matches the canonical bytes of the index.json using the public key in keys/official.pub.

• Algorithm: Ed25519
• Key ID: Must match between the signature envelope and the public key file.
• Payload: The SHA-256 hash of the canonical JSON bytes.

Sources: scripts/validate.py:73-95, README.md:90-96

Candidate Validation (Stage 5)

The validate_candidate.py script acts as a harness for validating new package specifications before they are merged into the main index. It handles the "Stage 5" intake process.

The harness performs a provisional merge of the candidate spec into a temporary index to test how it interacts with existing packages and the schema.
Sources: scripts/validate_candidate.py:35-42, 114-250, scripts/test_validate_candidate.py:166-200

Lifecycle Evidence Rules

For a package to pass validation, it must meet specific "activatable" criteria:

1. Plugins: Always require lifecycle evidence (verified_with field).
2. Modules: Require evidence only if they contain an activation section.
3. Scripts/Completions: Do not require evidence.

If evidence is required but missing, validation fails unless a --lifecycle-deferral reason is provided for staging environments.
Sources: scripts/validate_candidate.py:90-100, 203-220, scripts/validate.py:171-193

Safety and Security Invariants

The Validation Engine enforces several non-negotiable architectural invariants to maintain the trust root of the registry:

• No Private Keys: The engine explicitly scans for and fails if any *.key or *.pem material is detected in the source tree.
• Placeholder Signatures: Committed signatures in the main branch are often PLACEHOLDER. Real signatures are only generated in protected production workflows.
• Digest Immutability: Artifact digests (SHA-256) must be computed by the engine via add-package.py and never hand-typed.
• Workflow Pinning: All GitHub Actions must be pinned to a full 40-character commit SHA to prevent supply-chain attacks.

Sources: REVIEW.md:37-50, SECURITY.md:58-65, scripts/test_workflow_safety.py:44-53

Summary

The Validation Engine provides a robust, multi-layered defense mechanism that ensures every package in the Numan registry is structurally sound, cryptographically signed, and functionally verified against real Nushell environments. By mirroring production client logic in its Python tooling, it guarantees that the registry remains a reliable trust root for the ecosystem.