From 71fee220539430c4c14d729d7b72108de658c704 Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Fri, 14 Jun 2024 18:52:46 -0400 Subject: [PATCH 1/8] added oci registry readme help text Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/README.md b/README.md index 29fab318..170aaa72 100644 --- a/README.md +++ b/README.md @@ -92,6 +92,7 @@ Once idpbuilder finishes provisioning cluster and packages, you can access GUIs * ArgoCD: https://argocd.cnoe.localtest.me:8443/ * Gitea: https://gitea.cnoe.localtest.me:8443/ +#### Secrets You can obtain credentials for them by running the following command: ```bash @@ -227,6 +228,42 @@ You can also view the updated Application spec by going to this address: https:/ The second package directory defines two normal ArgoCD applications referencing a remote repository. They are applied as-is. +## Local OCI Registry + +The local Gitea instance created by idpbuilder contains a built in OCI registry for hosting container images as "packages" in Gitea nomenclature. + +It is a standard OCI registry, so the API should be compatible with any tools that are OCI compliant. That includes the `docker` cli. + +For example you can push an image by running: + +```bash +docker login gitea.cnoe.localtest.me:8443 +Username: giteaAdmin +Password: +docker push gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:with-app-fix2 +The push refers to repository [gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder] +78a0cd9d2976: Layer already exists +with-app-fix2: digest: sha256:50dc814b89e22988a69ac23aa7158daa834ab450b38b299e7f7fe17dba0ce992 size: 5566 +``` + +*NOTE: You cant get the giteaAdmin password in the same way as for the web or git interface.* + +```bash + ./idpbuilder get secrets -p gitea +``` + +### Pulling images internal to cluster: + +Because we are using an NGINX Ingress and pushing our image from off cluster, +Gitea and it's OCI registry think all images pushed to it are prefixed with `gitea.cnoe.localtest.me:8443`. + +This is correct by the OCI spec standards. However when you are on the cluster, that ingress is not available to you. +You can use the service name of gitea, but gitea will not know what images are being asked for at the svc domain name. + +So we use containerd to rewrite those image names so that they can be referenced at the external url: + +See `./pkg/kind/resources/kind.yaml.tmpl` for how this is done. + ## Contributing From 5cfe5b7fe4f8053efd6dd2ad6662552b2573543b Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Fri, 14 Jun 2024 19:11:59 -0400 Subject: [PATCH 2/8] note about this only working with domain based idpbuidler installations Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index 170aaa72..3a2c4761 100644 --- a/README.md +++ b/README.md @@ -251,6 +251,12 @@ with-app-fix2: digest: sha256:50dc814b89e22988a69ac23aa7158daa834ab450b38b299e7f ```bash ./idpbuilder get secrets -p gitea ``` +### Only Works With Subdomain based Idpbuilder installations +Right now because of the way the OCI registy specifications discovers information about a repo, this will only work with subdomain `gitea.cnoe.localtest.me` +based installations of idpbuilder's core capabilities. + +If you would like to use path based routing, you will have to install and manage your own OCI registry at this time. +Other registries might be able to handle this better, however which registries and how to configure them is beyond the scope of this readme. ### Pulling images internal to cluster: From b35c8305869c0823d698542f0ecc0b849a91049c Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Fri, 14 Jun 2024 19:20:55 -0400 Subject: [PATCH 3/8] add info about pulling Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/README.md b/README.md index 3a2c4761..1a708731 100644 --- a/README.md +++ b/README.md @@ -251,6 +251,70 @@ with-app-fix2: digest: sha256:50dc814b89e22988a69ac23aa7158daa834ab450b38b299e7f ```bash ./idpbuilder get secrets -p gitea ``` + +### Pulling Images + +You can pull an image back to your local machine using your docker client like so: + +``` +docker push gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder +Using default tag: latest +latest: Pulling from giteaadmin/beacon.idpbuilder +Digest: sha256:6308ebbce176470277dcca5e59aee3d528d9798a19f13d6a73ddd74a3f5da17b +Status: Downloaded newer image for gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:latest +gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:latest +``` + +### Referencing images in manifests on the idpbuilder cluster +If you are creating a pod or a deployment of some sort, you can refernce the images on the cluster using the same image name and tag like in the following example: + +``` +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: +spec: + progressDeadlineSeconds: 600 + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app: backstage + strategy: + rollingUpdate: + maxSurge: 25% + maxUnavailable: 25% + type: RollingUpdate + template: + metadata: + creationTimestamp: null + labels: + app: backstage + spec: + containers: + - command: + - node + - packages/backend + - --config + - config/app-config.yaml + env: + - name: LOG_LEVEL + value: debug + - name: NODE_TLS_REJECT_UNAUTHORIZED + value: "0" + envFrom: + - secretRef: + name: backstage-env-vars + - secretRef: + name: gitea-credentials + - secretRef: + name: argocd-credentials + image: gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:with-app-fix2 + imagePullPolicy: IfNotPresent +... + + + ### Only Works With Subdomain based Idpbuilder installations Right now because of the way the OCI registy specifications discovers information about a repo, this will only work with subdomain `gitea.cnoe.localtest.me` based installations of idpbuilder's core capabilities. From 577394f4696224f42861fdfbda001b159dd10e98 Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Mon, 17 Jun 2024 14:18:16 -0400 Subject: [PATCH 4/8] added one line for login to gitea Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index 1a708731..9b6c4142 100644 --- a/README.md +++ b/README.md @@ -252,6 +252,12 @@ with-app-fix2: digest: sha256:50dc814b89e22988a69ac23aa7158daa834ab450b38b299e7f ./idpbuilder get secrets -p gitea ``` +Or you can use this one liner to login: + +```bash +idpbuilder get secrets -p gitea -o json | jq '.[0].data.password' -r | docker login -u giteaAdmin --password-stdin gitea.cnoe.localtest.me:8443 +``` + ### Pulling Images You can pull an image back to your local machine using your docker client like so: From 27b6dc113bfe0772febc7fdfd1039ab1d6e22b6e Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Mon, 17 Jun 2024 14:22:06 -0400 Subject: [PATCH 5/8] fixed spelling and added oci registry spec info Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9b6c4142..e8bdc0a9 100644 --- a/README.md +++ b/README.md @@ -246,7 +246,7 @@ The push refers to repository [gitea.cnoe.localtest.me:8443/giteaadmin/beacon.id with-app-fix2: digest: sha256:50dc814b89e22988a69ac23aa7158daa834ab450b38b299e7f7fe17dba0ce992 size: 5566 ``` -*NOTE: You cant get the giteaAdmin password in the same way as for the web or git interface.* +*NOTE: You can't get the giteaAdmin password in the same way as for the web or git interface.* ```bash ./idpbuilder get secrets -p gitea @@ -272,7 +272,7 @@ gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:latest ``` ### Referencing images in manifests on the idpbuilder cluster -If you are creating a pod or a deployment of some sort, you can refernce the images on the cluster using the same image name and tag like in the following example: +If you are creating a pod or a deployment of some sort, you can reference the images on the cluster using the same image name and tag like in the following example: ``` apiVersion: apps/v1 @@ -322,12 +322,15 @@ spec: ### Only Works With Subdomain based Idpbuilder installations -Right now because of the way the OCI registy specifications discovers information about a repo, this will only work with subdomain `gitea.cnoe.localtest.me` +Right now because of the way the OCI registry specifications discovers information about a repo, this will only work with subdomain `gitea.cnoe.localtest.me` based installations of idpbuilder's core capabilities. If you would like to use path based routing, you will have to install and manage your own OCI registry at this time. Other registries might be able to handle this better, however which registries and how to configure them is beyond the scope of this readme. +For more info on the OCI registry spec and the root cause of this "discovery" issue see the spec here: +https://specs.opencontainers.org/distribution-spec/?v=v1.0.0#checking-if-content-exists-in-the-registry + ### Pulling images internal to cluster: Because we are using an NGINX Ingress and pushing our image from off cluster, From 370acc74ef40c44fa6ce6c4d87da44da9a4cd697 Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Mon, 17 Jun 2024 14:25:37 -0400 Subject: [PATCH 6/8] added note about no pull secrets needed Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index e8bdc0a9..d87c627d 100644 --- a/README.md +++ b/README.md @@ -319,7 +319,8 @@ spec: imagePullPolicy: IfNotPresent ... - +### No Pull Secret Needed +Our gitea instance allows for anonymous read access. This means that you can pull git repo contents and container images without the need to login. ### Only Works With Subdomain based Idpbuilder installations Right now because of the way the OCI registry specifications discovers information about a repo, this will only work with subdomain `gitea.cnoe.localtest.me` @@ -343,7 +344,6 @@ So we use containerd to rewrite those image names so that they can be referenced See `./pkg/kind/resources/kind.yaml.tmpl` for how this is done. - ## Contributing If you'd like to contribute to the project or know the architecture and internals of this project, check out the [contribution doc](./CONTRIBUTING.md). From e95b2e1a3a04eb021c9d1ab07261ec5cabe33fe5 Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Mon, 17 Jun 2024 14:28:34 -0400 Subject: [PATCH 7/8] fixd formatting issue Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d87c627d..51240fce 100644 --- a/README.md +++ b/README.md @@ -317,7 +317,7 @@ spec: name: argocd-credentials image: gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:with-app-fix2 imagePullPolicy: IfNotPresent -... +``` ### No Pull Secret Needed Our gitea instance allows for anonymous read access. This means that you can pull git repo contents and container images without the need to login. From 34ff44004ef4943c0aa1818ea6270d334a4fadb5 Mon Sep 17 00:00:00 2001 From: Jesse Sanford <108698+jessesanford@users.noreply.github.com> Date: Mon, 17 Jun 2024 14:32:16 -0400 Subject: [PATCH 8/8] more formatting Signed-off-by: Jesse Sanford <108698+jessesanford@users.noreply.github.com> --- README.md | 40 ++++------------------------------------ 1 file changed, 4 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index 51240fce..524576a0 100644 --- a/README.md +++ b/README.md @@ -271,7 +271,7 @@ Status: Downloaded newer image for gitea.cnoe.localtest.me:8443/giteaadmin/beaco gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:latest ``` -### Referencing images in manifests on the idpbuilder cluster +### Referencing Images In Manifests On The Idpbuilder K8s Cluster If you are creating a pod or a deployment of some sort, you can reference the images on the cluster using the same image name and tag like in the following example: ``` @@ -280,49 +280,17 @@ kind: Deployment metadata: annotations: spec: - progressDeadlineSeconds: 600 - replicas: 1 - revisionHistoryLimit: 10 - selector: - matchLabels: - app: backstage - strategy: - rollingUpdate: - maxSurge: 25% - maxUnavailable: 25% - type: RollingUpdate template: - metadata: - creationTimestamp: null - labels: - app: backstage spec: containers: - - command: - - node - - packages/backend - - --config - - config/app-config.yaml - env: - - name: LOG_LEVEL - value: debug - - name: NODE_TLS_REJECT_UNAUTHORIZED - value: "0" - envFrom: - - secretRef: - name: backstage-env-vars - - secretRef: - name: gitea-credentials - - secretRef: - name: argocd-credentials - image: gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:with-app-fix2 + - image: gitea.cnoe.localtest.me:8443/giteaadmin/beacon.idpbuilder:with-app-fix2 imagePullPolicy: IfNotPresent ``` ### No Pull Secret Needed Our gitea instance allows for anonymous read access. This means that you can pull git repo contents and container images without the need to login. -### Only Works With Subdomain based Idpbuilder installations +### Only Works With Subdomain Based Idpbuilder Installations Right now because of the way the OCI registry specifications discovers information about a repo, this will only work with subdomain `gitea.cnoe.localtest.me` based installations of idpbuilder's core capabilities. @@ -332,7 +300,7 @@ Other registries might be able to handle this better, however which registries a For more info on the OCI registry spec and the root cause of this "discovery" issue see the spec here: https://specs.opencontainers.org/distribution-spec/?v=v1.0.0#checking-if-content-exists-in-the-registry -### Pulling images internal to cluster: +### Pulling Images From Inside Idpbuilder K8s Cluster: Because we are using an NGINX Ingress and pushing our image from off cluster, Gitea and it's OCI registry think all images pushed to it are prefixed with `gitea.cnoe.localtest.me:8443`.