helm mode: add workflow for clustermesh via upgrade

[asauber]

Add a GitHub Actions Workflow for Clustermesh via Helm implementation of cilium upgrade

Test procedure used during local development

We can use the new upgrade command to enable Cluster Mesh on a set of two Cilium clusters with the following procedure.

Create a kind cluster with the following config

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: c1
nodes:
  - role: control-plane
  - role: worker
  - role: worker
  - role: worker
networking:
  disableDefaultCNI: true
  podSubnet: "10.10.0.0/16"
  serviceSubnet: "10.50.0.0/16"

kind create cluster --config c1.kindconfig.yaml

Test that the kind cluster came up with k get pods -A

Install Cilium using the CLI, making sure to set a Cluster ID and Cluster Name

export CILIUM_CLI_MODE=helm
cilium install --help
# note the suggestion to use --helm-set flags
cilium install --helm-set cluster.id=1 --helm-set cluster.name=cluster1

Create a second cluster using kind with the following config

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: c2
nodes:
  - role: control-plane
  - role: worker
  - role: worker
  - role: worker
networking:
  disableDefaultCNI: true
  podSubnet: "10.11.0.0/16"
  serviceSubnet: "10.51.0.0/16"

kind create cluster --config c2.kindconfig.yaml

Check that you are authed for Cluster kind-c2

Run k get pods -A and look for kube-apiserver-c2-control-plane

Install Cilium using the CLI, making sure to set a Cluster ID and Cluster Name

export CILIUM_CLI_MODE=helm
cilium install --help
# note the suggestion to use --helm-set flags
cilium install --helm-set cluster.id=2 --helm-set cluster.name=cluster2

Enable clustermesh on Cluster 1 using the new Helm upgrade command

export CILIUM_CLI_MODE=helm
cilium upgrade --help
# note the suggestion to use --helm-set flags
cilium upgrade --context $CLUSTER1 \
    --helm-set clustermesh.useAPIServer=true \
    --helm-set clustermesh.apiserver.service.type=NodePort
# observe that the clustermesh apiserver is running
k get pods -A

Prep environment for the clustermesh enable.

export CLUSTER1=kind-c1 CLUSTER2=kind-c2

Extract the CA cert from Cluster 1 and install it into Cluster 2, then restart Cilium on Cluster 2

kubectl --context $CLUSTER2 delete secret -n kube-system cilium-ca && \
kubectl --context $CLUSTER1 get secrets -n kube-system cilium-ca -oyaml \
    | kubectl --context $CLUSTER2 apply -f -
# you should see the following output
# secret "cilium-ca" deleted
# secret/cilium-ca created
# proceed to restart Cilium on kind-c2
kubectl --context $CLUSTER2 delete pod -l app.kubernetes.io/part-of=cilium -A

Enable clustermesh on Cluster 2 using the new Helm upgrade command

export CILIUM_CLI_MODE=helm
cilium upgrade --help
# note the suggestion to use --helm-set flags
cilium upgrade --context $CLUSTER2 \
    --helm-set clustermesh.useAPIServer=true \
    --helm-set clustermesh.apiserver.service.type=NodePort

Move the secrets expected by the cilium clustermesh connect command (this quirk to be fixed with a helm-mode PR for that command)

k get secrets --context $CLUSTER1 -n kube-system clustermesh-apiserver-remote-cert \
    -oyaml \
    | sed 's/name: .*/name: clustermesh-apiserver-client-cert/' \
    | k apply --context $CLUSTER1 -f -

k get secrets --context $CLUSTER2 -n kube-system clustermesh-apiserver-remote-cert \
    -oyaml \
    | sed 's/name: .*/name: clustermesh-apiserver-client-cert/' \
    | k apply --context $CLUSTER2 -f -

Connect the two clusters using ClusterMesh

cilium clustermesh connect --context $CLUSTER1 --destination-context $CLUSTER2

Run the Multi-Cluster connectivity tests.

cilium connectivity test --context $CLUSTER1 --multi-cluster $CLUSTER2

[michi-covalent]

@brlbil ping!

[michi-covalent]

is the button green? yes.