Implement registries for Kubernetes backend #4092
Conversation
* For each step that matches an registry, create a secret with the same name as its pod * Delete this same secret when the pod is deleted
qwerty287
added
enhancement
|
Please also update the docs :) |
There was a problem hiding this comment.
- Am I correct, that only one image pull secret is supported?
- There was
registryfield in UI, as I remember. You don't use it, right?
https://woodpecker-ci.org/docs/usage/registries#images-from-private-registries
Woodpecker matches the registry hostname to each image in your YAML. If the hostnames match, the registry credentials are used to authenticate to your registry and pull the image.
Seems, registries are checked on the Server. As you create/delete secret each time, it should work as intended and comments above are invalid.
However, I still believe, that creating one secret with all available registries/auths and then "attach" it (via imagePullSecret) to needed steps/pods is a better approach.
Can do, I wasn't sure if the docs site was live at HEAD or if there was some separate release process for them.
In Kubernetes? No, any number of secrets is supported, with any number of auths in each one . In woodpecker, I saw that the AuthConfig for a step is a single user:pass pair, which is matched against the list of repo, org, and global registries, so in that sense, a single extra secret is supported. That said, in the future, if sidecars are also supported, it may make sense for this one secret to contain multiple auths. Because you can have as many imagePullSecrets as you want in a pod, it makes much more sense to list the user-managed ones and then the agent-managed ones, instead of reading them from the API server and writing them back out, as the kubelet will do that for you, and retry against each one in sequence in the event of authentication errors. |
* Updated docs to remove unsupported functionality note and fix grammar * Use term "registry secret" in source to refer to temporary secrets created to contain registry authentication * Added separate env's to add extra labels and annotations to registry secrets * Consolidated registry secret functions into secrets.go * Moved secret creation/cleanup logic to separate functions in secret.go, called in kubernetes.go * Fixed whitespace errors and unused fields in test code
|
Tearing down https://woodpecker-ci-woodpecker-pr-4092.surge.sh |
There was a problem hiding this comment.
Overall, looks very well to me.
Am I correct, that only one image pull secret is supported?
There was registry field in UI, as I remember. You don't use it, right?
I was confused by flashbacks from #3122. I thought, if registry isn't passed within AuthConfig, then how Kubernetes would choose right secret? Or the only one secret/registry would be supported.
Then I reread through my PR
Made the secrets pipeline-wide instead of step-wide as in Docker backend
and understood everything :) ⬇️
registries are checked on the Server. As you create/delete secret each time, it should work as intended
--
if sidecars are also supported, it may make sense for this one secret to contain multiple auths
Good catch.
instead of reading them from the API server and writing them back out
Don't get this. Who reads regcreds from API? Kubernetes API? What is the "back"?
kubelet will do that for you, and retry against each one
That is exactly, what I had in mind, implementing #3122. ⬇️
However, I still believe, that creating one secret with all available registries/auths and then "attach" it (via imagePullSecret) to needed steps/pods is a better approach.
Yeah, maybe even not to "needed" pods, but all of them. And let Kubernetes cope with that.
There was a problem hiding this comment.
As it is tested and works (#4092 (comment)), no more comments from me.
Thank you for the contribution 🎉
According to the documentation, per-organization and per-pipeline registries are currently unsupported for the Kubernetes backend.
This patch implements this missing functionality by creating and deleting a matching secret for each pod with a matched registry, using the same name, labels, and annotations as the pod, and appending it to its
imagePullSecretslist.This patch adds tests for the new functionality, and has been manually end-to-end-tested in KinD by using a private image hosted in the matching gitea instance.
This will require updating the matching helm charts to add the create/delete permissions to the agent role, which is already done.
close #2987