Garbage Collection (GC)
Relevant source files
The following files were used as context for generating this wiki page:
Garbage Collection (GC) in Harbor is a system-level background service responsible for reclaiming storage space by removing unreferenced blobs and dangling manifests from the registry backend. It identifies artifacts that are no longer linked to any repository or project and purges them from both the database and the physical storage.
The process is designed to be compute-intensive and can impact registry performance, which is why it supports scheduling, manual triggers, and a "Dry Run" mode to preview potential space savings without performing actual deletions.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:121-160, src/portal/src/i18n/lang/en-us-lang.json:1152-1155
Architecture and Execution Flow
The GC process is managed by the Harbor Jobservice and consists of two primary phases: Mark and Sweep.
Mark Phase
In the Mark phase, the system identifies candidates for deletion. It scans the ArtifactTrash table and examines blobs to determine if they are referenced by any existing manifests. It also checks for untagged artifacts if the delete_untagged parameter is enabled. Candidates are added to a deleteSet.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:171-224
Sweep Phase
The Sweep phase performs the actual removal. It iterates through the deleteSet and utilizes parallel workers to delete manifests via the Registry V2 API and blobs via the Registry Controller client. After physical deletion, it removes the corresponding records from the Harbor database.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:226-343
Workflow Diagram
The following flowchart illustrates the high-level logic from the initial trigger to the final cleanup.
This diagram represents the transition from identifying orphan artifacts to physical cleanup and subsequent system updates.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:138-168, src/controller/gc/callback.go:44-54
Component Logic
Garbage Collector Job
The GarbageCollector struct implements the job.Interface. It handles initialization, parameter parsing, and execution management. It uses an errgroup.Group to manage concurrent workers during the sweep phase.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:62-85, src/jobservice/job/impl/gc/garbage_collection.go:245-255
API and Controller
The GC Controller manages the lifecycle of GC executions and schedules. It provides endpoints for creating, updating, and stopping GC tasks.
| Endpoint | Method | Description |
|---|---|---|
/system/gc/schedule | POST/PUT | Creates or updates the GC schedule (Cron). |
/system/gc | GET | Retrieves GC execution history. |
/system/gc/{gc_id} | GET | Gets the status of a specific GC job. |
/system/gc/{gc_id} | PUT | Stops a running GC job. |
/system/gc/{gc_id}/log | GET | Retrieves logs for a specific GC task. |
| Sources: src/server/v2.0/handler/gc.go:40-150, api/v2.0/swagger.yaml:1040-1150 |
Configuration Parameters
GC can be tuned using the following parameters:
| Parameter | Type | Default | Description |
|---|---|---|---|
delete_untagged | Boolean | true | Whether to delete artifacts that have no tags. |
delete_tag | Boolean | true | Whether to remove tag files from backend storage. |
dry_run | Boolean | false | If true, only estimates space without deleting. |
time_window | Integer | 2 | Hours; excludes artifacts uploaded recently. |
workers | Integer | 1 | Number of parallel workers (Max 10). |
| Sources: src/jobservice/job/impl/gc/garbage_collection.go:101-135, src/server/v2.0/handler/gc.go:214-220 |
Internal Data Flow
Manifest and Blob Deletion Sequence
Deleting a manifest requires additional care because Harbor must ensure the revisions folder in each repository is cleaned up.
The sequence ensures that storage is only removed after the database and manifest references are cleared.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:275-340
Post-GC Tasks
Once a GC job completes successfully, a callback is triggered to refresh project quotas. This is necessary because GC frees up physical space that should be reflected in the user's project usage metrics.
Sources: src/controller/gc/callback.go:44-54
Frontend Interaction
The Harbor Portal provides a user interface for administrators to trigger "GC Now", start a "Dry Run", or configure a Cron-based schedule (Hourly, Daily, Weekly, or Custom).
- Status Tracking: The UI polls the latest GC history to display the last completed time and current job status (e.g., Running, Success).
- Worker Configuration: Admins can select the number of parallel workers via the Portal configuration.
Sources: src/portal/src/app/base/left-side-nav/clearing-job/gc-page/gc/gc.component.ts:65-90, src/portal/src/app/base/left-side-nav/clearing-job/gc-page/gc/gc.component.ts:133-160
Implementation Details
Cache Invalidation
Harbor cleans the registry's Redis cache for blobs and repositories at the end of a non-dry-run GC job. This prevents the registry from attempting to serve cached metadata for blobs that no longer exist on disk.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:346-375
Stop Signal Handling
The GC job monitors for an operation command (OPCommand). If a stop signal is received, the job quits gracefully, ensuring the registry is not left in an inconsistent state, though it may attempt to clean the cache if some deletions already occurred.
Sources: src/jobservice/job/impl/gc/garbage_collection.go:151-155, src/jobservice/job/impl/gc/garbage_collection.go:453-459
Conclusion
Garbage Collection is a critical maintenance system within Harbor that ensures storage sustainability. By identifying orphan blobs through the Mark and Sweep pattern, and providing administrative controls for concurrency and dry runs, it allows system operators to manage backend growth while maintaining data integrity across the database and the registry filesystem.