Artifact Replication Engine
Relevant source files
The following files were used as context for generating this wiki page:
The Artifact Replication Engine is the core synchronization component of Harbor Satellite, responsible for maintaining the desired state of container images at edge locations. It orchestrates the flow of OCI artifacts from a central Harbor instance (source) to local registries (destination) by periodically fetching state definitions from Ground Control and executing replication tasks.
This engine ensures that edge environments remain operational even with intermittent connectivity by maintaining a local, integrity-checked copy of required images. It handles both the replication of new artifacts and the cleanup of obsolete ones based on the difference between the current local state and the desired remote state.
Sources: README.md:9-16, CLAUDE.md:120-125, internal/state/state_process.go:73-81
Core Architecture and Components
The replication process is managed by the FetchAndReplicateStateProcess, which coordinates multiple sub-processes to synchronize artifacts and configuration.
State Process Components
The engine relies on several key structures to manage the synchronization lifecycle:
| Component | Description |
|---|---|
FetchAndReplicateStateProcess | The primary orchestrator that manages state fetching, change detection, and replication execution. |
StateMap | A structure mapping specific state URLs to their current entities and reader interfaces. |
Artifact | The fundamental data unit representing an OCI image, including its repository, tags, and digest. |
Entity | A flattened representation of an artifact used for granular comparison (Repository + Name + Tag + Digest). |
Replicator | The interface responsible for the actual movement of bits between registries. |
Sources: internal/state/state_process.go:17-48, internal/state/artifact.go:17-25
Replication Workflow
The replication engine follows a structured execution path to ensure consistency:
The diagram shows the high-level flow of the replication engine, highlighting the transition from setup to parallel processing of artifact groups and system configurations.
Sources: internal/state/state_process.go:61-110
State Synchronization Logic
The engine determines which actions to take by comparing the "Desired State" (fetched from Ground Control) against the "Current State" (stored locally).
Change Detection
The GetChanges function performs a three-way comparison to identify entities that need to be replicated or deleted:
- New Entities: Present in the new state but missing from the old state.
- Updated Entities: Present in both, but with different digests.
- Removed Entities: Present in the old state but missing from the new state.
This sequence illustrates the interaction between the state process, the fetcher, and the replicator during a single synchronization cycle.
Sources: internal/state/state_process.go:281-325
Artifact and Group Management
Artifacts are organized into Groups, which are collections of container images that satellites replicate together.
Data Structures
The relationship between high-level Artifact definitions and the granular Entity objects used during the comparison phase.
Sources: internal/state/artifact.go:17-25, internal/state/state_process.go:374-386
Ground Control Integration
Ground Control facilitates replication by assembling "State Artifacts"—OCI images containing JSON payloads that describe the desired state.
- Group State: Assembled as
satellite/group-state/<group-name>/state:latest. - Satellite State: A root-level artifact containing a list of group states and a configuration pointer.
- Artifact Transformation: The engine ensures repository formats are correct by splitting the repository string into
repoandimagecomponents before processing.
Sources: ground-control/internal/utils/helper.go:88-91, ground-control/internal/utils/helper.go:173-175, internal/state/state_process.go:362-372
Execution Requirements and Configuration
Before replication can begin, the engine validates a set of mandatory parameters.
Validation Checks
The CanExecute function verifies the presence of:
satelliteStateURL: The source of truth for the satellite.remoteURL: The destination local registry.srcURL: The upstream source registry.srcUsername/srcPassword: Credentials for the source registry.
Sources: internal/state/state_process.go:193-219
Registry Mirroring
The engine can configure local container runtimes (Docker, containerd, CRI-O, Podman) to use the local registry as a mirror. This is handled via the --mirrors flag, which points specific registries to the local instance.
Sources: CLAUDE.md:127-135, internal/utils/utils.go:94-98
Conclusion
The Artifact Replication Engine provides a robust mechanism for edge artifact distribution by abstracting the complexities of OCI registry synchronization into a state-driven process. By leveraging parallel state fetching and granular entity comparison, it maintains local registry integrity while optimizing for bandwidth and connectivity constraints inherent in edge computing environments.
Sources: README.md:18-28, internal/state/state_process.go:221-230