Upstream Outreach
Relevant source files
The following files were used as context for generating this wiki page:
Upstream Outreach is a strategic module within the Numan Registry project designed to manage the transition of packages from registry-hosted mirrors to official upstream-provided assets. The primary goal is to encourage maintainers of Nushell plugins and modules to ship byte-stable release assets in Numan-supported formats (such as .zip, .tar.gz, or .tar.xz), thereby reducing the registry's reliance on custom mirrors.
The system encompasses a tracking mechanism for GitHub issues and pull requests, an automated synchronization script that updates project documentation, and a set of intake rules that prioritize upstream assets over auto-generated GitHub archive zipballs. This outreach is tightly integrated with the package intake lifecycle, ensuring that once an upstream maintainer ships a compliant asset, the registry can switch from a mirror provenance to an upstream provenance.
Sources: README.md:16-24, scripts/sync-intake-candidates.py:16-28
Outreach Lifecycle and Status Tracking
The outreach process is tracked through docs/intake-state.json, which contains metadata about candidates and their current outreach status. The system classifies the progress of contacting maintainers into several distinct states, ranging from "not started" to "upstream shipped."
Status Classification Logic
The system automatically determines the status of an outreach effort by querying the GitHub API using the gh CLI. It looks for issues authored by the registry maintainer in the upstream repository that match specific search terms like "numan" or "release zip."
The diagram shows the logic used to determine the status of an outreach effort based on GitHub issue metadata.
Sources: scripts/sync-intake-candidates.py:126-218
Outreach Data Model
The outreach_status function derives a structured status object used for rendering documentation.
| Field | Description |
|---|---|
issue | A Markdown link to the GitHub issue or PR (e.g., [#123](https://github.com/numan-cli/numan-registry/blob/b01a4c780f52b8001310f2b0987e371fd7fba2f4/url)). |
opened | The ISO date (YYYY-MM-DD) when the outreach issue was created. |
response | Indicates if an upstream maintainer responded (e.g., yes (2024-01-01) or awaiting). |
upstream_shipped | A boolean-style indicator showing if the upstream project now provides official assets. |
summary | A human-readable string summarizing the current state (e.g., "issue open, awaiting response"). |
Sources: scripts/sync-intake-candidates.py:126-150
Synchronization and Documentation
The sync-intake-candidates.py script serves as the primary engine for updating outreach documentation. It generates and updates two main files: docs/intake-candidates.md and docs/upstream-release-outreach.md.
Automated Documentation Updates
The sync process ensures that the human-readable tracking table in the outreach document stays current with the actual state of the registry index and GitHub interactions.
- Registry Switching: The system detects when a package has successfully transitioned. If a package exists in the
liveregistry index and no longer uses a mirror URL, it is marked asswitched: yes. - Provenance Detection: The script classifies artifact URLs into
mirror,ci-built, orupstreamto determine if outreach is still required.
The sequence shows how the sync script orchestrates data from local JSON files and external GitHub state to update the outreach tracker.
Sources: scripts/sync-intake-candidates.py:348-410, scripts/render_catalog_compat.py:65-92
Intake Rules and Provenance
Outreach is governed by specific "Intake Rules" defined in the project documentation. These rules dictate how the registry should handle different types of artifacts and how the transition to upstream assets is prioritized.
Priority and Guidelines
- Supported Formats: Artifacts must use supported archive suffixes (
.zip,.tar.gz,.tar.xz). - Asset Preference: The registry MUST prefer upstream-uploaded release assets over GitHub's auto-generated
/archive/zipballs, which are not byte-stable. - Mirroring as Fallback: If no compliant upstream asset exists, a mirror zip is built from a tag/commit snapshot using
scripts/build-mirror-zip.pyand hosted by the registry. - Hash Verification: SHA256 hashes must never be hand-typed; they must be computed via
scripts/add-package.pyduring the download process.
Sources: scripts/sync-intake-candidates.py:314-320, README.md:126-135
Provenance Classification
The system uses heuristics to classify the origin of artifacts, which directly impacts the outreach tracker.
def artifact_provenance(url: str) -> str:
"""Classify a pinned artifact URL as mirror, ci-built, upstream, or other."""
if "numan-registry/releases/download/mirror-" in url:
return "mirror"
if "tonythethompson/numan-plugins/releases/" in url:
return "ci-built"
if url.startswith("http"):
return "upstream"
return "other"
Sources: scripts/sync-intake-candidates.py:44-52
Outreach Tools and Metadata
Several scripts support the generation of candidates and the validation of their metadata before they are promoted to the registry.
Candidate Generation
scripts/gen_candidate.py transforms discovery reports into draft registry specs. It includes a _meta block that records the provenance of fields, which is used by maintainers to verify what information was discovered versus what was guessed.
Validation and Evidence
scripts/validate_candidate.py acts as a harness to ensure that the candidate's metadata is sound. It checks for:
- Duplicate intake PRs using
_warn_open_intake_prs. - Structural linting via
scripts/lint_packages.py. - Schema validation.
- Optional lifecycle proof via
scripts/lifecycle-prove.py(Stage 1 acceptance).
Sources: scripts/gen_candidate.py:182-205, scripts/validate_candidate.py:160-185
Summary
Upstream Outreach is a critical workflow for maintaining the integrity and sustainability of the Numan Registry. By automating the tracking of GitHub issues and providing clear guidelines for artifact provenance, the project ensures that it can effectively collaborate with upstream maintainers. The goal remains a transition toward official, byte-stable upstream assets, minimizing the registry's role as a mirror host while maximizing the security and reliability of the Nushell ecosystem.