Build Matrix & Cross Compilation
Relevant source files
The following files were used as context for generating this wiki page:
The build matrix and cross-compilation system in numan-plugins serves as a CI/CD pipeline feeder for the numan registry. Its primary purpose is to bridge the gap for Nushell plugins that only ship source code by providing signed, hash-pinned binary artifacts for multiple architectures and operating systems. The system cross-compiles plugins from immutable upstream commits, packages them into normalized archives, and prepares them for registry intake.
Sources: README.md:1-17, manifest.json:2-4
The architecture relies on GitHub Actions runners (native and cross-toolchains) to produce binaries for x86_64 and aarch64 across Linux, macOS, and Windows. This ensure that users do not need a Rust toolchain locally to install popular plugins like nu_plugin_highlight or nu_plugin_regex.
Sources: README.md:40-44, manifest.json:161-167
Build Matrix Configuration
The build matrix is defined in manifest.json, which maps Rust target triples to specific GitHub Actions runner labels. The system supports five primary targets by default.
Supported Targets and Runners
| Rust Target Triple | GitHub Runner Label | Compilation Method |
|---|---|---|
x86_64-unknown-linux-gnu | ubuntu-latest | Native |
aarch64-unknown-linux-gnu | ubuntu-latest (cross) | Cross-compilation |
x86_64-apple-darwin | macos-15-intel | Native |
aarch64-apple-darwin | macos-15 | Native |
x86_64-pc-windows-msvc | windows-latest | Native |
Sources: manifest.json:5-12, manifest.json:160-168, scripts/test_workflow_safety.py:120-134
Target Selection Logic
The validate_manifest.py script manages the calculation of required targets for each plugin. It combines global defaults with plugin-specific exclusions.
The diagram shows the logic used to determine which binaries must be built for a specific plugin entry.
Sources: scripts/validate_manifest.py:44-67
Cross-Compilation Implementation
Cross-compilation is specifically utilized for aarch64-unknown-linux-gnu targets. This is achieved using the taiki-e/setup-cross-toolchain-action within the GitHub Actions workflow. Other targets, including macOS aarch64, utilize native runners.
Sources: README.md:42-44, manifest.json:163
Known Limitations and Exclusions
Certain plugins exclude specific targets due to compilation environment constraints:
- OpenSSL/TLS Issues:
nu_plugin_prometheusexcludesaarch64-unknown-linux-gnubecause theopenssl-syscrate fails to locate target headers under the cross-toolchain. - Dependency Mismatches:
nu_plugin_dnsexcludes Windows due toCargo.lockconflicts andnu-protocoltype mismatches. - Runner Availability: Many plugins previously excluded
x86_64-apple-darwindue tomacos-13runner stalls, requiring a transition tomacos-15-intel.
Sources: manifest.json:44-46, manifest.json:104-106, manifest.json:14-16
Packaging and Determinism
Once binaries are built, they are packaged into platform-specific archives. To ensure security and hash-pinning reliability, the packaging process is strictly deterministic.
Archive Specifications
- Unix (Linux/macOS): Compressed
.tar.gzformat. - Windows:
.zipformat. - Permissions: Executables are set to
0o755. - Timestamps: All file
mtimevalues are fixed to1980-01-01 UTC(315532800) to ensure identical binary output results in an identical archive hash.
Sources: scripts/package_plugin.py:15-32, scripts/package_plugin.py:40-62
Packaging Workflow
The sequence shows how the packaging script transforms a raw binary into a standardized, deterministic distribution artifact.
Sources: scripts/package_plugin.py:73-100
Specification Generation
The final step in the build process is generating a registry-compatible specification using gen_spec.py. This script aggregates results from all matrix jobs and manifest metadata.
Data Flow for Spec Generation
- Manifest Input: Reads plugin metadata (version, owner, description, upstream repo).
- Source Provenance: Pins the immutable
source_commitassource.revto maintain a clear trust boundary. - Target Aggregation: Collects the
PACKAGEDoutput from every matrix job to map target triples to their respective release URLs. - Verification: Ensures that every expected target (calculated by
validate_manifest.py) is present in the final asset set.
Sources: scripts/gen_spec.py:1-25, scripts/gen_spec.py:108-142
Technical Constraints
| Element | Requirement | Source |
|---|---|---|
sha256 | Omitted from final spec (registry re-hashes on intake) | scripts/gen_spec.py:12 |
source.rev | Must be a 40-character lowercase hex SHA | scripts/validate_manifest.py:103 |
release_base | Base URL for GitHub release assets | scripts/gen_spec.py:128 |
Sources: scripts/gen_spec.py:1-152, scripts/validate_manifest.py:1-110
Summary
The Build Matrix & Cross Compilation system provides a robust pipeline for transforming Rust source code into verifiable binaries. By combining strict manifest validation, deterministic packaging, and targeted cross-compilation for ARM64 Linux, the project enables automated, secure delivery of Nushell plugins across the major computing architectures.