kubeadm init hangs on ppc64le

[krishvoor]

Facing the below issue ..
SELINUX is disabled.
OS : Ubuntu 1604
ARCH : ppc64le
iptables/firewalld : disabled
Natively compiled Kubernetes (release 1.5) and tried " kubeadm init ". Hangs here :-

[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[preflight] Running pre-flight checks
[preflight] WARNING: kubelet service does not exist
[init] Using Kubernetes version: v1.5.2
[certificates] Generated Certificate Authority key and certificate.
[certificates] Generated API Server key and certificate
[certificates] Generated Service Account signing keys
[certificates] Created keys and certificates in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[apiclient] Created API client, waiting for the control plane to become ready

@luxas

[luxas]

You should compile the hyperkube image for ppc64le and then run

KUBE_HYPERKUBE_IMAGE=your-image-here kubeadm init

The thing is that there has to be a docker image built for ppc64le that can be used for bootstrapping the control plane. Right now there are no images for ppc64le to use; hence the hanging kubeadm; it waits for kubelet to start images that don't exist.

[krishvoor]

Here is where I'm stuck at from past 30 minutes..
KUBE_HYPERKUBE_IMAGE=gcr.io/google_containers/hyperkube-ppc64le:v1.3.0-alpha.3 kubeadm init
[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[preflight] Running pre-flight checks
[preflight] WARNING: kubelet service does not exist
[init] Using Kubernetes version: v1.5.2
[certificates] Generated Certificate Authority key and certificate.
[certificates] Generated API Server key and certificate
[certificates] Generated Service Account signing keys
[certificates] Created keys and certificates in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[apiclient] Created API client, waiting for the control plane to become ready

[luxas]

You have to use v1.4+ for kubeadm, build hyperkube fron the PR I linked

[krishvoor]

I believe this should suffice . as this is >1.4+
kubeadm version
kubeadm version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.3061+14e322cc827ca5-dirty", GitCommit:"14e322cc827ca546eb80b721a825a43311a99437", GitTreeState:"dirty", BuildDate:"2017-01-11T09:04:34Z", GoVersion:"go1.8beta2", Compiler:"gc", Platform:"linux/ppc64le"}

[luxas]

I'm talking about the image of k8s you're really using: 'KUBE_HYPERKUBE_IMAGE=gcr.io/google_containers/hyperkube-ppc64le` is now version v1.3.0-alpha.3 and you should build a hyperkube image with version v1.4-HEAD from this PR: kubernetes/kubernetes#38926

[krishvoor]

Apologizes for the delayed response. Here is what I have done to create the respective docker image :-

/opt/gopath/src/github.com/kubernetes/cluster/images/hyperkube# git branch

• pr/38926
  release-1.5
  /opt/gopath/src/github.com/kubernetes/cluster/images/hyperkube#

I had to make couple of changes to Makefile as well Dockerfile to get "ARCH=ppc64le make build VERSION=1.4" working.

docker images gcr.io/google_containers/hyperkube-ppc64le
REPOSITORY TAG IMAGE ID CREATED SIZE
gcr.io/google_containers/hyperkube-ppc64le 1.4 f436f732dbeb 28 minutes ago 637 MB

However from last 20 minutes I'm stuck at same point

KUBE_HYPERKUBE_IMAGE=gcr.io/google_containers/hyperkube-ppc64le:1.4 kubeadm init
[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[preflight] Running pre-flight checks
[preflight] WARNING: kubelet service does not exist
[init] Using Kubernetes version: v1.5.2
[certificates] Generated Certificate Authority key and certificate.
[certificates] Generated API Server key and certificate
[certificates] Generated Service Account signing keys
[certificates] Created keys and certificates in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[apiclient] Created API client, waiting for the control plane to become ready

[krishvoor]

I have finally figured out a way to get it working, followed this approach to setup Master as well Worker node

https://github.com/kubernetes/kube-deploy/tree/master/docker-multinode

One observation was, it was looking for a docker images gcr.io/google_containers/hyperkube-ppc64le:v1.5.2 and also assumes etcd and flanneld are already in place.

That said, thank you for your help. And I'm closing this issue.

[mkumatag]

@harsha544 the issue was with missing gcr.io/google_containers/etcd-ppc64le:3.0.17 image. Now image is available in the repo, can you try now and let me know your findings with upstream kubeadm.

[krishvoor]

@mkumatag Thanks for the response, I shall give it a try and let you know :)

[timothysc]

@harsha544 is this still an issue or can we close now?

[krishvoor]

I'll close this. I'll work with Manju incase i find any difficulties.