Package Intake Process
Relevant source files
The following files were used as context for generating this wiki page:
The Package Intake Process is a multi-stage workflow designed to discover, validate, and incorporate new Nushell packages into the official Numan registry. The process ensures that all entries meet strict criteria for security, platform compatibility, and functional integrity through automated scanning and manual review gates.
This process spans from initial discovery (Stage 3) to the final opening of an intake Pull Request (Stage 6). It relies on a suite of Python scripts to automate artifact hashing, schema validation, and lifecycle testing against real Nushell environments. Successful intake culminates in the promotion of a package to the signed registry/index.json catalog.
Sources: README.md:126-150, scripts/validate_candidate.py:1-20
Workflow Architecture
The intake process follows a linear progression of stages, moving from raw discovery data to a production-ready registry entry.
The diagram illustrates the sequential stages of the package intake lifecycle.
Sources: scripts/validate_candidate.py:10-20, scripts/open_intake_pr.py:1-10
Stage 4: Candidate Generation
In Stage 4, discovery reports (generated in Stage 3) are transformed into draft registry specs. The gen_candidate.py script maps asset filenames to known target triples (e.g., x86_64-pc-windows-msvc) and determines package types such as plugins, modules, or scripts.
Key logic includes:
- Target Mapping: Uses regex patterns to identify architecture and OS from filenames.
- Provenance Metadata: Includes a
_metablock to track which fields were discovered versus guessed. - Defaulting: Defaulting
nu_versionto*if no constraint is found inCargo.toml.
Sources: scripts/gen_candidate.py:1-14, scripts/gen_candidate.py:18-35
Stage 5: Validation and Proof
Stage 5 acts as a validation harness. The script validate_candidate.py orchestrates several sub-processes to ensure a candidate is viable before any Git mutations occur.
| Step | Script/Tool | Description |
|---|---|---|
| Download & Hash | add-package.py | Fetches artifacts and computes SHA-256 digests. |
| Structural Lint | lint_packages.py | Checks for naming conventions and required fields. |
| Schema Validation | validate.py | Verifies the candidate against schemas/index-v1.json. |
| Lifecycle Proof | lifecycle-prove.py | (Opt-in) Installs and runs the package against a real Nu binary. |
Sources: scripts/validate_candidate.py:10-20, scripts/validate_candidate.py:157-190
Stage 6: Intake Pull Request
Once a candidate passes validation, open_intake_pr.py automates the creation of a GitHub Pull Request. This involves creating a feature branch (e.g., intake/owner-name-version), updating the temporary index, and generating a detailed PR body containing evidence from Stage 5.
The sequence shows the automated steps for finalizing a package intake submission.
Sources: scripts/open_intake_pr.py:246-278, scripts/open_intake_pr.py:281-300
Artifact and Provenance Rules
The intake process enforces strict rules regarding how artifacts are handled to maintain registry integrity:
- Byte-Stable Hashes: Artifact digests must come from
scripts/add-package.pyvia actual download. Hand-typing SHA-256 values is forbidden (P0 violation). - Supported Formats: Only
.zip,.tar.gz, and.tar.xzare supported for intake. - Provenance Classification:
upstream: Direct release assets from original authors.mirror: Registry-hosted archives for tags/commits lacking release assets.ci-built: Plugins built via thenuman-pluginspipeline.
- No Private Keys: Secret scanning must pass for every intake; private keys (e.g.,
*.key,*.pem) must never be committed.
Sources: REVIEW.md:35-50, scripts/sync-intake-candidates.py:46-60, README.md:100-110
Intake State Management
The status of packages currently in the pipeline is tracked in docs/intake-state.json and rendered into docs/intake-candidates.md.
| State Section | Meaning |
|---|---|
| ready | Upstream ships compliant release assets. |
| mirror | Requires build-mirror-zip.py for registry hosting. |
| blocked | Package has identified issues preventing intake. |
| changelog | Historical record of intake activities. |
Sources: scripts/sync-intake-candidates.py:245-300, scripts/open_intake_pr.py:126-150
Conclusion
The Package Intake Process ensures that the Numan registry remains a trustworthy source of Nushell extensions. By requiring automated validation, lifecycle proof for activatable packages, and mandatory human review via Pull Requests, the system balances rapid catalog growth with rigorous security and compatibility standards. All production catalog changes must pass through this pipeline, ensuring that every SHA-256 digest in the signed index is verified and provenance is clearly documented.
Sources: README.md:126-150, REVIEW.md:75-90