Utility Scripts
Relevant source files
The following files were used as context for generating this wiki page:
Utility scripts in the Harbor Satellite project provide the automation necessary for local development, end-to-end (E2E) testing, environment provisioning, and binary compilation. These scripts bridge the gap between core Go logic and the infrastructure required to run the Satellite and Ground Control components, such as PostgreSQL databases, Harbor registries, and SPIRE servers.
The scope of these utilities includes automated setup scripts for different SPIFFE attestation methods, local integration test runners, and a centralized Task-based build system. They ensure that complex multi-component workflows—like registering a satellite with a join token or configuring OCI registry mirrors—can be executed reliably across different environments.
Local Test Automation
The project includes specialized shell scripts to orchestrate the entire lifecycle of the Harbor Satellite system on a local machine. These scripts manage dependencies, environment variables, and component registration.
Local Test Orchestration
The local-test.sh script automates the deployment of a development environment. It performs a sequential setup of the database, Ground Control service, and the Satellite itself.
- Dependency Management: Checks for and starts a PostgreSQL container if not already running.
- Environment Provisioning: Generates a
.envfile for Ground Control with database and Harbor credentials. - Service Execution: Starts Ground Control and the Satellite as background processes.
- API Integration: Performs REST calls to Ground Control to create test groups, configurations, and register the satellite to obtain an authentication token.
Sources: scripts/local-test.sh:37-185
Automation Flow
The following diagram illustrates the sequence of operations performed by the local test utility to bootstrap the system.
Sources: scripts/local-test.sh:188-217
Deployment and Setup Scripts
For SPIFFE-enabled environments, the project provides modular setup scripts that handle the complexities of identity issuance and workload registration.
SPIFFE Attestation Setup
Different scripts exist for various attestation methods (e.g., Join Token, X.509 PoP). These scripts manage the SPIRE (SPIFFE Runtime Environment) infrastructure.
- Certificate Generation: Utilities to create bootstrap CA certificates and X.509/SSH proof-of-possession credentials.
- Workload Registration: Automates the creation of SPIRE entries using selectors (e.g., Docker labels) to assign identities to the Ground Control and Satellite containers.
- Health Verification: Implements retry loops to ensure SPIRE servers and agents are healthy before proceeding with service startup.
Sources: deploy/quickstart/spiffe/join-token/external/gc/setup.sh:22-110, deploy/quickstart/spiffe/x509pop/external/gc/setup.sh:14-95
Key Setup Parameters
The setup scripts utilize several environment variables and configuration files to define the environment:
| Parameter | Source | Description |
|---|---|---|
HARBOR_URL | .env / Script Var | The endpoint of the upstream Harbor registry. |
GC_TOKEN | SPIRE CLI | Join token generated by SPIRE server for agent attestation. |
agent-gc-runtime.conf | Generated File | SPIRE agent configuration containing the trust_domain and join_token. |
GC_AGENT_ID | SPIRE CLI | Dynamically extracted SPIFFE ID for the attested agent. |
Sources: deploy/quickstart/spiffe/join-token/external/gc/setup.sh:45-84, deploy/quickstart/spiffe/x509pop/external/gc/setup.sh:62-75
Build and E2E Task Automation
The project uses Taskfile.yml to define a high-level interface for build and test operations, abstracting complex go build and docker commands.
Build Utilities
The build system supports cross-platform compilation for both Satellite and Ground Control.
- Satellite: Compiled from
./cmd/main.go. - Ground Control: Compiled from
./ground-control/main.go. - Matrix Builds: The
build.ymlutility supports a matrix ofGOOS(linux, darwin) andGOARCH(amd64, arm64, ppc64le, s390x, riscv64).
Sources: taskfiles/build.yml:7-80
E2E Test Suite
The e2e.yml taskfile manages the lifecycle of end-to-end tests, including Harbor initialization and verification of image replication.
Sources: taskfiles/e2e.yml:19-216
Internal Go Utilities
The project includes a utils package in Go that provides programmatic helpers used by the main components and scripts for file and network operations.
- Path Validation:
GetAbsFilePathandHasInvalidPathCharsensure configuration paths are safe and reachable. - Registry Formatting:
FormatRegistryURLstandardizes URLs by stripping protocol prefixes (http://orhttps://). - Network Helpers:
ValidateRegistryAddressvalidates IP addresses (supporting IPv4 only) and port ranges. - Signal Handling:
SetupContextcreates a cancellable context that responds toSIGTERMandSIGINTfor graceful shutdowns.
Sources: internal/utils/utils.go:39-93
Summary of Utility Components
| Component | Files | Primary Purpose |
|---|---|---|
| Local Runner | local-test.sh | Rapid local development and testing without full containerization. |
| SPIFFE Setup | setup.sh (various) | Provisioning SPIRE and mTLS identities for zero-trust environments. |
| Task Runner | Taskfile.yml, taskfiles/*.yml | Centralized entry point for building, linting, and E2E testing. |
| Go Utils | utils.go, helper.go | Shared logic for file I/O, URL parsing, and Harbor API interactions. |
Sources: Taskfile.yml:18-50, internal/utils/utils.go:1-125, scripts/local-test.sh:1-20
Utility scripts in Harbor Satellite serve as the glue between the distributed components. By automating infrastructure tasks like database migrations, SPIRE agent attestation, and Harbor replication policy creation, these scripts allow developers to focus on core artifact distribution logic while ensuring a consistent environment across local and CI/CD platforms.