Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harbor as a cross-cloud global registry for the Kubernetes Project #14411

Open
hh opened this issue Mar 10, 2021 · 4 comments
Open

Harbor as a cross-cloud global registry for the Kubernetes Project #14411

hh opened this issue Mar 10, 2021 · 4 comments
Assignees
Labels
kind/requirement New feature or idea on top of harbor

Comments

@hh
Copy link

hh commented Mar 10, 2021

Is your feature request related to a problem? Please describe.

The #k8s-infra-wg exists within the Kubernetes community to support the infrastructure underlying the project.

Currently the Google donated cloud-credits are largely spent on k8s.gcr.io as the primary registry container artefacts are pulled from there. It's on the order of 50-60% of the total spend.

This cost should be distributed by allowing caches or distributed mirror solutions that can be run locally by kubernetes providers / clouds and picked up by users when deploying Kubernetes and pulling the k8s.io released OCI images.

Describe the solution you'd like

As Harbor is also a CNCF project, I think it would be a great solution, creating a cohesive Cloud Native story for registry.k8s.io as a global image/artefact distribution.

Ideally we have a solution that points to registries hosted at Google, Microsoft, Amazon, and a few others. Possibly using their local cloud specific registries, but having a clear option to deploy Harbor in a best practice and community supported manner.

Describe the main design/architecture of your solution

It's an ongoing discussion, which is evolving but some initial ideas:

  • Maintain a mapping of Autonomous System Numbers for cloud providers networks that have a local mirror/cache registry.
  • Use the SRC ip to redirect (via HTTP/302 or split-horizon DNS) to cloud local mirror
  • Dynamically modifying the manifest.json to redirect to cloud local mirror (may have token issues, as most registries do not allow you to pull these blobs without auth)

Describe the development plan you've considered

  • Deploy harbor.packet.k8s.io on Packet
  • Deploy harbor.microsoft.k8s.io on Azure (or a registry hosted by them)
  • Deploy split-horizon DNS PoC : registry-dns.k8s.io
  • Deploy http redirect proxy PoC : registry-302.k8s.io
  • Show the two PoCs pulling images locally using the two

I have two engineers at ii.coop (@hh and @BobyMCbobs) and an evolving team at Microsoft that are keen to help.

Additional context

Some initial conversations in the various slacks:

@BobyMCbobs initial explorations for Possible implementations for registry.k8s.io:

@hh
Copy link
Author

hh commented Mar 10, 2021

Related: Declarative Deployment of Helm - goharbor/harbor-helm#884

@dims
Copy link

dims commented Mar 24, 2021

cc @justaugustus

@hh
Copy link
Author

hh commented Mar 24, 2021

Lot's of good links over here as well: distribution/distribution#3379 (comment)

@github-actions
Copy link

github-actions bot commented Jul 5, 2022

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

@github-actions github-actions bot added the Stale label Jul 5, 2022
@stonezdj stonezdj removed the Stale label Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/requirement New feature or idea on top of harbor
Projects
None yet
Development

No branches or pull requests

6 participants