Intake State Tracking

Relevant source files

The following files were used as context for generating this wiki page:

• scripts/sync-intake-candidates.py
• scripts/open_intake_pr.py
• scripts/validate_candidate.py
• README.md
• REVIEW.md
• scripts/test_sync_intake_candidates.py

Intake State Tracking is a systematic process within the Numan Registry designed to manage the lifecycle of packages from initial discovery and evaluation as candidates to their eventual publication in the official registry. This system ensures that all packages—whether they are plugins, modules, or scripts—undergo rigorous validation, including artifact provenance checks and lifecycle evidence gathering, before being promoted to the production catalog.

The system relies on a central state file, docs/intake-state.json, which acts as the machine-readable source of truth for the intake pipeline. Automation scripts synchronize this state with human-readable documentation and GitHub activity (such as Pull Requests and Outreach Issues) to provide a transparent view of the registry's growth and compliance status.

Sources: scripts/sync-intake-candidates.py:17-21, README.md:27-37

Architecture and Data Flow

The intake process is governed by a series of scripts that transition a package through various stages. The data flow starts with the definition of a candidate in intake-state.json, proceeds through automated validation, and results in a GitHub Pull Request for maintainer review.

Intake Lifecycle Stages

The project defines several stages for package intake:

1. Candidate Discovery: Packages are identified and added to intake-state.json.
2. Validation (Stage 5): The scripts/validate_candidate.py script downloads artifacts, computes hashes, and runs structural lints.
3. Lifecycle Evidence (Stage 1): Activatable packages must prove compatibility with a real Nushell binary using scripts/lifecycle-prove.py.
4. PR Generation (Stage 6): scripts/open_intake_pr.py assembles a branch with the spec and validation evidence to open a registry intake PR.

Core Components

Intake State Synchronization

The sync-intake-candidates.py script is responsible for maintaining consistency between the machine-readable state and the project documentation. It classifies packages based on their provenance: mirror, ci-built, upstream, or mixed.

Validation Harness

The validation harness (validate_candidate.py) ensures that a candidate spec is structurally sound and that its artifacts are retrievable. It performs the following checks:

• Download and Hash: Verifies artifact URLs and computes SHA-256 digests.
• Structural Lint: Runs lint_packages.py to ensure compliance with registry standards.
• Schema Validation: Ensures the spec matches the required JSON schema.
• Lifecycle Proof: Optionally runs the package against a real Nushell environment.

Sources: scripts/validate_candidate.py:115-180

Data Models

Intake State Schema

The docs/intake-state.json file uses a specific structure to categorize packages.

Section	Description
ready	Packages that ship byte-stable release assets and are ready for index inclusion.
mirror	Packages requiring a registry-hosted mirror due to lack of stable upstream assets.
blocked	Packages that have been evaluated but are currently excluded due to specific blockers.
changelog	A record of historical changes to the intake state.

Sources: scripts/sync-intake-candidates.py:250-320

Package Provenance Categories

Artifacts are classified by their source to determine outreach and mirroring requirements.

Category	Definition
mirror	Artifacts hosted at numan-registry/releases/download/mirror-.
ci-built	Artifacts built by tonythethompson/numan-plugins.
upstream	Official release assets from the original maintainer's repository.
mixed	A version containing targets from multiple provenance types.

Sources: scripts/sync-intake-candidates.py:44-53, scripts/test_sync_intake_candidates.py:33-46

Operational Invariants

The intake system enforces strict safety and trust invariants:

• No Hand-typed Hashes: SHA-256 values must always be computed from a download via scripts/add-package.py or the validation harness.
• Provenance Preservation: CI-built plugins must retain their immutable upstream commit (source.rev) and tag provenance.
• Evidence Mandatory: Activatable packages (plugins and certain modules) require successful lifecycle-prove evidence before promotion to production.
• Placeholder Signatures: The source tree index.json.sig is always a PLACEHOLDER; real signing occurs only in protected deployment workflows.

Sources: REVIEW.md:32-48, README.md:20-25

Summary

Intake State Tracking provides the rigorous framework necessary to maintain a high-trust registry for Nushell packages. By combining automated validation scripts (validate_candidate.py, open_intake_pr.py) with transparent state tracking (intake-state.json), the project ensures that every package in the official catalog is verified, compatible, and secure.