Proxy Cache & Pull-through
Relevant source files
The following files were used as context for generating this wiki page:
Proxy Cache is a feature in Harbor that allows a project to act as a pull-through cache for a remote registry instance. When a user requests an artifact from a proxy cache project, Harbor checks its local storage; if the artifact is missing, it transparently pulls it from the configured remote registry, stores it locally, and serves it to the user.
This system optimizes network bandwidth by reducing repeated downloads from external registries and provides high availability by serving cached content even when the upstream registry is temporarily unavailable. It supports various registry types, including DockerHub, Harbor, AWS ECR, and Azure ACR. Sources: README.md, src/portal/src/i18n/lang/en-us-lang.json, tests/apitests/python/test_proxy_cache.py:27-42
Architecture and Request Flow
The Proxy Cache logic is primarily implemented as middleware that intercepts GET and HEAD requests for manifests and blobs. It determines whether to serve an artifact from the local repository or fetch it from a remote source.
Manifest Request Handling
When a manifest is requested, the system first checks if it can proxy for the project. It then attempts to locate the manifest locally. If it must fetch from remote, it handles tag associations and digest matching to ensure the local cache remains consistent with the upstream. Sources: src/server/middleware/repoproxy/proxy.go:169-231
The flow above describes the decision-making process when a client requests an image manifest from a proxy-enabled project. Sources: src/server/middleware/repoproxy/proxy.go:177-228
Blob Request Handling
Blob requests follow a similar pattern but utilize specialized readers and io.CopyN to manage memory efficiency when dealing with large image layers. The system also verifies content digests to ensure integrity. Sources: src/server/middleware/repoproxy/proxy.go:64-111
Key Components
| Component | Description |
|---|---|
| Proxy Middleware | Intercepts registry V2 API calls and redirects them to proxy logic if the project is configured as a cache. |
| Connection Limiter | A Redis-backed mechanism to restrict the number of concurrent connections to an upstream registry. |
| Remote Helper | An interface that handles communication with external registries, supporting speed limits (bandwidth throttling). |
| Controller Instance | Coordinates between local storage lookups and remote fetching operations. |
Sources: src/server/middleware/repoproxy/proxy.go:49-62, src/server/middleware/repoproxy/proxy.go:195-200
Configuration and Metadata
Proxy Cache behavior is controlled through project-level metadata and system-wide configuration settings.
Project-Level Settings
When creating or updating a project, specific fields in ProjectMetadata define proxy behavior:
registry_id: The ID of the remote registry that this project proxies.proxy_speed_kb: Bandwidth limit in Kbps for communication between Harbor and the upstream registry.max_upstream_conn: Maximum concurrent connections per artifact to the upstream registry (set to -1 for no limit).proxy_cache_local_on_not_found: If "true", Harbor serves cached content even if the artifact is removed from upstream or the upstream is unreachable.
Sources: api/v2.0/swagger.yaml:2630-2655, src/portal/src/i18n/lang/en-us-lang.json
Permitted Registry Types
Harbor restricts which registry types can be used for proxy caching. The default permitted types include:
docker-hub, harbor, azure-acr, aws-ecr, google-gcr, quay, docker-registry, github-ghcr, jfrog-artifactory. Sources: src/pkg/chart/testdata/harbor-schema2/templates/core/core-cm.yaml:57
Security and Access Control
Proxy cache projects implement strict access rules to maintain the integrity of the cache.
Pushing Restrictions
By default, standard user sessions cannot push artifacts directly to a project configured as a proxy cache. Only specific "proxy sessions" (used by internal services or scanners generating SBOMs) are allowed to perform write operations to ensure the cache only contains verified upstream content. Sources: src/server/middleware/repoproxy/proxy.go:264-282
The sequence diagram illustrates the protection mechanism preventing external users from polluting the pull-through cache. Sources: src/server/middleware/repoproxy/proxy.go:283-298
Tag Association
After a HEAD request for a manifest by a client (e.g., Docker 20.10+), Harbor asynchronously ensures the tag is associated with the digest in the local cache. This ensures that even if an image is pulled by digest, the tag remains current in the proxy cache. Sources: src/server/middleware/repoproxy/proxy.go:300-322
Summary
The Proxy Cache and Pull-through system in Harbor provides a transparent layer between container runtimes and remote registries. By leveraging Redis-based connection limiting and project-level metadata, it balances performance, bandwidth usage, and reliability. It effectively turns a Harbor project into a resilient gateway for external dependencies, while strictly controlling write access to maintain cache purity. Sources: src/server/middleware/repoproxy/proxy.go, api/v2.0/swagger.yaml