Architecture & Data Flow

Relevant source files

The following files were used as context for generating this wiki page:

• README.md
• REVIEW.md
• manifest.json
• scripts/package_plugin.py
• scripts/gen_spec.py
• scripts/release_transaction.py
• scripts/validate_manifest.py
• scripts/ensure_release_absent.py

Introduction

The numan-plugins repository serves as a CI build and delivery pipeline for Nushell plugins. Its primary purpose is to cross-compile source-only plugins into signed, hash-pinned binary artifacts for the numan registry. The system acts as a "feeder" that bridge the gap between source-only upstream repositories and the requirements of the numan installer, which necessitates prebuilt release binaries for users who lack a full Rust toolchain.

Sources: README.md:1-12, README.md:14-16

The architecture is built around a "Trust Boundary" where this repository handles binary building and hosting, while official signing keys and the trust root remain in the separate numan-registry repository. Data flows from immutable upstream commits, through a multi-platform build matrix, and finally into a "Draft-then-finalize" release transaction that ensures asset integrity before public consumption.

Sources: README.md:20-27, REVIEW.md:22-24

System Architecture

The project utilizes a centralized manifest to drive a distributed build and packaging process. The architecture focuses on immutability and provenance tracking.

Core Components

Component	Description
manifest.json	The source of truth containing active plugins, upstream pins (source_commit), and target runner mappings.
validate_manifest.py	Enforces schema requirements and verifies that upstream tags resolve to specified immutable commits.
package_plugin.py	Normalizes binaries into deterministic .tar.gz (Unix) or .zip (Windows) archives with fixed timestamps.
gen_spec.py	Generates a numan-registry compatible kind: binary specification, including source provenance metadata.
release_transaction.py	Manages the lifecycle of a GitHub release (Claim -> Upload -> Finalize) to prevent partial releases.

Sources: README.md:30-41, scripts/package_plugin.py:11-15, scripts/gen_spec.py:4-15

Build Matrix and Target Mapping

The system supports cross-compilation for five primary targets using a mix of native runners and setup-cross-toolchain-action.

Data Flow: Plugin Lifecycle

The lifecycle of a plugin build flows from manifest configuration to the handoff of a generated specification to the registry.

1. Validation and Pre-flight

Before any build begins, the system validates the manifest.json. The validate_manifest.py script ensures all required fields (e.g., repo, source_commit, nu_version) are present and that the source_commit is a valid 40-character SHA. Crucially, it uses git ls-remote to verify that the human-facing tag provided in the manifest matches the immutable source_commit.

Sources: scripts/validate_manifest.py:15-30, scripts/validate_manifest.py:113-125

2. The Build-Release Transaction

To ensure release integrity, the system uses a transactional approach managed by release_transaction.py.

3. Packaging and Spec Generation

The package_plugin.py script creates deterministic archives. It uses a FIXED_MTIME (1980-01-01) and specific Unix modes to ensure that rebuilding the same source produces an identical sha256 hash.

Sources: scripts/package_plugin.py:30-33, scripts/package_plugin.py:65-75

The gen_spec.py script collects PACKAGED output lines (containing target, filename, and local sha256) to build the final registry specification.

Sources: scripts/gen_spec.py:61-75, scripts/gen_spec.py:126-147

Trust Boundary and Security Invariants

The architecture maintains strict security invariants to ensure the integrity of the binaries delivered to users.

1. Builds only, never signs: This repository hosts binaries; signing keys (Ed25519) stay in numan-registry.
2. No private keys: Private material is prohibited from the repository.
3. Immutable Upstream Pins: Every plugin must pin a source_commit.
4. Release Immutability: The workflow refuses to overwrite existing tags or assets via ensure_release_absent.py.
5. External Hash Computation: While gen_spec.py uses hashes for internal sanity checks, the numan-registry re-downloads and re-hashes assets during intake to avoid trusting hand-typed values.

Sources: REVIEW.md:21-36, README.md:20-27, scripts/ensure_release_absent.py:18-35

Data Models

Manifest Entry Schema

The manifest.json contains a list of active plugins. Each entry must conform to the following structure:

Field	Type	Description
repo	String	Upstream GitHub repository (owner/repo).
source_commit	String	40-character immutable commit SHA.
tag	String	Human-facing provenance tag.
plugin_bin	String	The name of the resulting binary.
nu_version	String	Semver range for Nushell compatibility.
exclude_targets	Array	(Optional) Targets where this plugin does not build.

Sources: scripts/validate_manifest.py:18-30, manifest.json:20-35

Specification Data Flow

The output of the pipeline is a spec-<name>.json file. This is the primary data handoff to the numan-registry.

Conclusion

The numan-plugins architecture is a specialized binary delivery pipeline that prioritizes immutability and deterministic builds. By using a manifest-driven approach combined with transactional release management, the system ensures that Nushell plugins are packaged safely and with clear provenance before being consumed by the official registry.

Sources: README.md:20-27, REVIEW.md:21-31