Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubekins-e2e-v2 multi arch container image for ppc64le #34345

Open
Rajalakshmi-Girish opened this issue Feb 17, 2025 · 6 comments
Open

kubekins-e2e-v2 multi arch container image for ppc64le #34345

Rajalakshmi-Girish opened this issue Feb 17, 2025 · 6 comments
Labels
sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra.

Comments

@Rajalakshmi-Girish
Copy link
Contributor

Rajalakshmi-Girish commented Feb 17, 2025

This is to have a kubekins-e2e-v2 image for ppc64le as per the discussion here https://kubernetes.slack.com/archives/CCK68P2Q2/p1739538851706299?thread_ts=1739359083.466129&cid=CCK68P2Q2

Ref: https://github.com/kubernetes/test-infra/tree/master/images/kubekins-e2e-v2

@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Feb 17, 2025
@Rajalakshmi-Girish
Copy link
Contributor Author

Below are the findings when we tried building image using Dockerfile defined here

Following tools need support for ppc64le :

  • cfssl
  • kind
  • kubetest2

Also, there is below error while running /google-cloud-sdk/install.sh in Dockerfile.
ERROR: (gcloud.components.update) The following components are unknown [gcloud-crc32c]

@Rajalakshmi-Girish
Copy link
Contributor Author

/sig k8s-infra

@k8s-ci-robot k8s-ci-robot added sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Feb 17, 2025
@dims
Copy link
Member

dims commented Feb 17, 2025

@Rajalakshmi-Girish can you please try to add a if condition to skip gcloud installer thingy and see if you can make progress.

@Rajalakshmi-Girish
Copy link
Contributor Author

[root@raji-ws kubekins-e2e-v2]# git diff
diff --git a/images/kubekins-e2e-v2/Dockerfile b/images/kubekins-e2e-v2/Dockerfile
index caa854205c..d3d681e63c 100644
--- a/images/kubekins-e2e-v2/Dockerfile
+++ b/images/kubekins-e2e-v2/Dockerfile
@@ -66,11 +66,13 @@ ARG GCLOUD_SDK_URL=https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud
 RUN wget -O google-cloud-sdk.tar.gz -q $GCLOUD_SDK_URL && \
     tar xzf google-cloud-sdk.tar.gz -C / && \
     rm google-cloud-sdk.tar.gz && \
-    /google-cloud-sdk/install.sh \
-    --disable-installation-options \
-    --bash-completion=false \
-    --path-update=false \
-    --usage-reporting=false && \
+    if [ "${TARGETARCH}" != "ppc64le" ]; then \
+        /google-cloud-sdk/install.sh \
+        --disable-installation-options \
+        --bash-completion=false \
+        --path-update=false \
+        --usage-reporting=false; \
+    fi && \
     gcloud components install alpha beta && \
     gcloud info | tee /workspace/gcloud-info.txt

@@ -128,11 +130,13 @@ RUN mkdir /docker-graph
 # The invocation at the end is to prevent download failures downloads as in the bug.
 # TODO(porridge): bump CFSSL_VERSION to one where cfssljson supports the -version flag and test it as well.
 ARG CFSSL_VERSION
-RUN wget -q -O cfssl "https://github.com/cloudflare/cfssl/releases/download/v${CFSSL_VERSION}/cfssl_${CFSSL_VERSION}_linux_${TARGETARCH}" && \
+RUN if [ "${TARGETARCH}" != "ppc64le" ]; then \
+    wget -q -O cfssl "https://github.com/cloudflare/cfssl/releases/download/v${CFSSL_VERSION}/cfssl_${CFSSL_VERSION}_linux_${TARGETARCH}" && \
     wget -q -O cfssljson "https://github.com/cloudflare/cfssl/releases/download/v${CFSSL_VERSION}/cfssljson_${CFSSL_VERSION}_linux_${TARGETARCH}" && \
     chmod +x cfssl cfssljson && \
     mv cfssl cfssljson /usr/local/bin && \
-    cfssl version
+    cfssl version; \
+fi

 # replace kubectl with one from K8S_RELEASE
 ARG K8S_RELEASE=latest
[root@raji-ws kubekins-e2e-v2]#

I was able to build the image after skipping the gcloud install.sh and cfssl sections.
The install of KIND and KUBETEST2 were not attempted as the variables KIND_VERSION and KUBETEST2_VERSION were not passed.

[root@raji-ws kubekins-e2e-v2]# docker build . --build-arg GO_VERSION=1.24.0 --build-arg YQ_VERSION=v4.40.7 --no-cache
[+] Building 117.1s (23/23) FINISHED                                                                                                          docker:default
 => [internal] load build definition from Dockerfile                                                                                                    0.0s
 => => transferring dockerfile: 7.01kB                                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/debian:bookworm                                                                                      0.1s
 => [internal] load .dockerignore                                                                                                                       0.0s
 => => transferring context: 2B                                                                                                                         0.0s
 => [internal] load build context                                                                                                                       0.0s
 => => transferring context: 91B                                                                                                                        0.0s
 => [ 1/18] FROM docker.io/library/debian:bookworm@sha256:4abf773f2a570e6873259c4e3ba16de6c6268fb571fd46ec80be7c67822823b3                              0.0s
 => CACHED [ 2/18] WORKDIR /workspace                                                                                                                   0.0s
 => [ 3/18] RUN mkdir -p /workspace                                                                                                                     0.9s
 => [ 4/18] RUN apt-get update && apt-get install -y --no-install-recommends     build-essential     ca-certificates     curl     file     git     ip  40.1s
 => [ 5/18] RUN wget -O google-cloud-sdk.tar.gz -q https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud-sdk.tar.gz &&     tar xzf google-clo  15.2s
 => [ 6/18] RUN apt-get update && apt-get install -y --no-install-recommends     apt-transport-https     ca-certificates     curl     gnupg2     soft  17.4s
 => [ 7/18] RUN curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg       1.6s
 => [ 8/18] RUN apt-get update &&     apt-get install -y --no-install-recommends docker-ce docker-buildx-plugin &&     rm -rf /var/lib/apt/lists/* &&  10.7s
 => [ 9/18] RUN echo 'DOCKER_OPTS="${DOCKER_OPTS} --data-root=/docker-graph"' |     tee --append /etc/default/docker                                    1.2s
 => [10/18] RUN mkdir /docker-graph                                                                                                                     1.6s
 => [11/18] RUN if [ "ppc64le" != "ppc64le" ]; then     wget -q -O cfssl "https://github.com/cloudflare/cfssl/releases/download/v${CFSSL_VERSION}/cfss  1.9s
 => [12/18] RUN rm -f $(which kubectl) &&     export KUBECTL_VERSION=$(curl -L https://dl.k8s.io/release/latest.txt) &&     wget https://dl.k8s.io/rel  2.8s
 => [13/18] RUN wget -q "https://go.dev/dl/go1.24.0.linux-ppc64le.tar.gz" &&     tar xzf "go1.24.0.linux-ppc64le.tar.gz" -C /usr/local &&     mkdir -p  5.4s
 => [14/18] RUN wget -q "https://github.com/mikefarah/yq/releases/download/v4.40.7/yq_linux_ppc64le"     -O /usr/local/bin/yq && chmod +x /usr/local/b  1.7s
 => [15/18] RUN if [ -n "${KIND_VERSION}" ]; then     wget -q -O /usr/local/bin/kind https://kind.sigs.k8s.io/dl/v${KIND_VERSION}/kind-linux-ppc64le &  1.5s
 => [16/18] RUN if [ -n "${KUBETEST2_VERSION}" ]; then     wget -q https://storage.googleapis.com/k8s-staging-kubetest2/latest/linux-ppc64le.tgz &&     1.5s
 => [17/18] RUN [ -n "https://mirror.gcr.io" ] &&     echo "DOCKER_OPTS="${DOCKER_OPTS} --registry-mirror=https://mirror.gcr.io"" |     tee --append /  1.3s
 => [18/18] COPY [runner.sh,     /usr/local/bin/]                                                                                                       0.0s
 => exporting to image                                                                                                                                 12.2s
 => => exporting layers                                                                                                                                12.1s
 => => writing image sha256:1d05e11d16f025539b0dbf2c1a9dd91f605f4348384d953361775414c9d2a150                                                            0.0s
[root@raji-ws kubekins-e2e-v2]# 

@Rajalakshmi-Girish
Copy link
Contributor Author

@dims we have submitted a request to kubetest2 to support ppc64le kubernetes-sigs/kubetest2#289
I would appreciate your suggestion on kind and cfssl, please.

@dims
Copy link
Member

dims commented Feb 19, 2025

@Rajalakshmi-Girish the diff above looks reasonable to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra.
Projects
None yet
Development

No branches or pull requests

3 participants