minikube_k8s_Readme.md

WSL2 image with Minikube installation

This image is based on WSL2 docker image. Pleae make sure you have that image or have followed the process to create that image. All the steps in this documentation assumes that all packages and softwares are installed as in docker WSL2 image.

Motivation

To have a WSL image that can create Kubernetes (K8s) setups as required by certain applications or a custom setup. There are different versions available that help setup Kubernetes. For this image we use MiniKube.

Why MiniKube?

Frankly, there is no preference, as I am not aware of all the nitty-gritties of kubernetes and various flovours of it. I selected minikube as it seems on of the default version for development.I also hope that if I face issues, minikube may have a more active and wide community support. This can then be used as a base image for other applications, that may have similar setups based on minikube.

Create a new WSL environment

I use docker image as a starting point as Docker seems to be the most popular containerization platform. Minikube has support for other container and VM manager as well, but then it would need another component to learn and configure. Execute below commands to create starting WSL image to develop minikube WSL2 image

wsl --import deb11mk .\wsl\deb11mk\ .\wsl_backup\deb11_docker.tar

One must change name of images and paths based on the specific setup.

Install minikube

  # Find latest minikube version or set a version that you want to install
  MK_VERSION=$(curl -fsSL -o /dev/null -w "%{url_effective}" https://github.com/kubernetes/minikube/releases/latest | xargs basename)

  # FIND machine's CPU architecture. Currenttly Kind supports ARM and AMD
  ARCHITECTURE=$(dpkg --print-architecture)

  # Download minkube on the system 

  MK=minikube-linux-${ARCHITECTURE}

  curl -LO https://storage.googleapis.com/minikube/releases/${MK_VERSION}/${MK}

  
  # install minikube executable
  sudo install ${MK} /usr/local/bin/minikube

Install kubectl

There are different ways to install kubectl. Below is the procedure using binaries

NOTE: For some reason curl -L -s did not work for me with dl.k8s.io for K8S_VERSION, so I changed it to direct URL

# kubectl version

K8S_VERSION=$(curl -Ls https://storage.googleapis.com/kubernetes-release/release/stable.txt)

# Download Kubectl
curl -LO "https://dl.k8s.io/release/${K8S_VERSION}/bin/linux/${ARCHITECTURE}/kubectl"

# Install Kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Install Helm

Install Helm to create and manage Kubernetes deployments.

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3

chmod 700 get_helm.sh
./get_helm.sh

Install cmctl

This tool is required for cert-manager to issue HTTPS certificates for DNS names.

ARCHITECTURE=$(dpkg --print-architecture)

CMCTL_VERSION=$(curl -fsSL -o /dev/null -w "%{url_effective}" https://github.com/cert-man
ager/cert-manager/releases/latest | xargs basename)

curl -sSLO https://github.com/cert-manager/cert-manager/releases/download/$CMCTL_VERSION/
cmctl-linux-$ARCHITECTURE.tar.gz

tar xzf cmctl-linux-$ARCHITECTURE.tar.gz

sudo mv cmctl /usr/local/bin

Configure autocompletion

In this image we have ZSH configured as default shell using Oh My Posh. Execute below commands to add autocompletion for Kind and kubectl for z sh.

cmctl : execute below commands

cmctl completion zsh > _cmctl
sudo mv _cmctl /usr/share/zsh/vendor-completions/_cmctl

kubectl,minikube and helm: Add plugins to .zshrc file for Oh My ZSH or execute similar commands as above, but for kubectl and helm

plugins =(
        #other plugins
        kubectl
        helm
        minikube
)

Install VS Code extensions

Install below extesions inside WSL VS Code server.

• Install Kubernetes extension in VS Code.
• Install Helm Intellisense

Create a basic cluster

minikube start

It works!!!! :happy:

If you have installed VS Code extensions above, then details of cluster would be visible in VS Code as well.

• In Kubernetes

• In Docker extension, as K8s is running in Docker after all

Problem

This container is having issues accessing https://k8s.gcr.io"!!

In the output above of minikube start you see this message. However, when I login in to container, I can reach to the repository. However, I cannot reach to registry.docker.io or registry-1.docker.io

docker exec -it minikube bash
root@minikube:/# ping k8s.gcr.io
PING googlecode.l.googleusercontent.com (74.125.133.82) 56(84) bytes of data.
64 bytes from wo-in-f82.1e100.net (74.125.133.82): icmp_seq=1 ttl=105 time=42.3 ms
root@minikube:/# ping docker.com
PING docker.com (54.198.192.151) 56(84) bytes of data.

root@minikube:/# ping docker.io
PING docker.io (44.197.137.126) 56(84) bytes of data.

root@minikube:/# traceroute registry-1.docker.io
registry-1.docker.io: No address associated with hostname
Cannot handle "host" cmdline arg `registry-1.docker.io` on position 1 (argc 1)

root@minikube:/# ping registry.docker.io
ping: registry.docker.io: Name or service not known

I logged out and logged in again to the minikube docker container. Now registry-1.docker.io works!! However, registry.docker.io still does not work

After yet another logout-login step it stopped working again. Not sure where is the issue.

root@minikube:/# ping registry.docker.com
PING registry.docker.com (205.251.197.9) 56(84) bytes of data.
64 bytes from ns-1289.awsdns-33.org (205.251.197.9): icmp_seq=1 ttl=241 time=24.4 ms
64 bytes from ns-1289.awsdns-33.org (205.251.197.9): icmp_seq=2 ttl=241 time=24.5 ms

root@minikube:/# ping registry-1.docker.io
PING registry-1.docker.io (174.129.220.74) 56(84) bytes of data.

root@minikube:/# ping registry.docker.io
ping: registry.docker.io: Name or service not known

I stopped minikube cluster, purged the data/cache and started the cluster again. This time I did not get a warning about k8s.gcr.io. But still it cannot reach registry.docker.io!

 minikube start
😄  minikube v1.25.1 on Debian 11.2 (amd64)
✨  Automatically selected the docker driver
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
🐳  Preparing Kubernetes v1.23.1 on Docker 20.10.12 ...
    ▪ kubelet.housekeeping-interval=5m
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

root@minikube:/# traceroute k8s.gcr.io
traceroute to k8s.gcr.io (142.250.13.82), 30 hops max, 60 byte packets
 1  host.minikube.internal (192.168.49.1)  0.032 ms  0.006 ms  0.004 ms

root@minikube:/# traceroute registry.docker.io
registry.docker.io: Temporary failure in name resolution
Cannot handle "host" cmdline arg `registry.docker.io` on position 1 (argc 1)

root@minikube:/# traceroute registry-1.docker.io
registry-1.docker.io: No address associated with hostname
Cannot handle "host" cmdline arg `registry-1.docker.io` on position 1 (argc 1)

FINAL Solution

After 2 weeks of trying to find solution, I almost gave up. Then one day I came back and started again same commands. This time registry.docker.io was accessible and I could contiue to work!!!

It probably was something related to docker setup in WSL as I got this solution working first in the Kind WSL image.

NOTE: I do not have Docker Desktop for Windows installed. Each WSl image has docker installed locally in it. Yet, somehow when kind image started working, minikube also was working!!

NO IDEA what solved it.

Memory usage: heavy

On my machine this basic cluster with 1 node and no actual pod running took about 7GB on my WSL2 VM. I do have another WSL2 image running with VS code, where this repository is being updated. At the same time minikube WSL2 image is also running with VS Code to test extensions and work on config.yaml etc.

Inside minikube WSL

Export the minikube image as WSL image

Now we have a basic minikube distribution/flavour of K8s running on this image. We can export this image and save it for later use as starting point.

1. Delete the temporary cluster

   minikube stop

2. Remove minikube container and all details

   minikube delete --purge

3. Remove any left over docker images

   docker image prune

4. Export the image as mk WSL image

   wsl --export deb11kindk8s .\wsl_backup\deb11_kind.tar | gzip.exe -9 .\wsl_backup\deb11_kind.tar

   Note that gzip is piped to compress the tar file. Later I found that Windows has inbuild tar and it worked faster than gzip.

   wsl --export deb11kindk8s .\wsl_backup\deb11_kind.tar | tar -czf .\wsl_backup\deb11_kind.tar.gz .\wsl_backup\deb11_kind.tar
   
   rm .\wsl_backup\deb11_kind.tar

   However, I don't know how to improve this command to avoid repeating filenames.

NOTE: Compressed image file is now about 600 MB.