added: support for adding labels from the config file itself

[yashvardhan-kukreja]

closes #1574

My approach:
So, the labelling of the nodes is going to be one of the actions (actionsToRun) when cluster creation happens in the code.
Now, I have not coded a dedicated action for "just the labelling of nodes".
I have rather created a file postcreationconfiguration.go which will consist of all the functions which are supposed to occur after the cluster gets created. (like labelling in our case)

And in the file postcreationconfiguration.go, I have written a labelNodes function which will label the nodes from the config file.

Why did I do it?
Because, say, in the future, there is another requirement like adding taints or tolerations (or some other configuration, which usually in real life are done after cluster is created) through the config file itself.
Then, we can write functions for those things in the postcreationconfigurations.go itself instead of creating dedicated actions for them. That's because these things like adding taints or tolerations are usually done after the cluster is already created.

What's happening behind the scenes?
So, basically, at every iteration, if the labels are being applied to a control-plane node, then, that control-plane-node itself will self-apply those labels to itself.
But if the labels are being applied to a worker node, then, one of the control plane nodes will apply those labels to that worker node.

Usage from user's side:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: control-plane
  labels:
    name: master
    index: 2
- role: worker
  labels:
    name: worker
    tier: app
- role: worker

Output: (Look over the end of every labels row)

NAME                   STATUS  ROLES   LABELS
master-control-plane   Ready   master  beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,index=2,kubernetes.io/arch=amd64,kubernetes.io/hostname=master-control-plane,kubernetes.io/os=linux,name=master,node-role.kubernetes.io/master=
master-control-plane2  Ready   master  beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master-control-plane2,kubernetes.io/os=linux,node-role.kubernetes.io/master=
master-worker          Ready   <none>  beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master-worker,kubernetes.io/os=linux
master-worker2         Ready   <none>  beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master-worker2,kubernetes.io/os=linux,name=worker,tier=app

[k8s-ci-robot]

Hi @yashvardhan-kukreja. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yashvardhan-kukreja
To complete the pull request process, please assign munnerz after the PR has been reviewed.
You can assign the PR to them by writing /assign @munnerz in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[BenTheElder]

Why wouldn't we have kubelet apply these?

[yashvardhan-kukreja]

Why wouldn't we have kubelet apply these?

Labels can be added to control plane nodes as well right in the config file.
then, how would we go on applying labels to control plane nodes?
If we go with kubelet's way of applying node labels, then, we'd have to code different separate logics for adding labels to worker nodes (using kubelet at the time of registering nodes) and adding labels to control-plane nodes (using kubectl or something else).

Also, here, in the code, the labels are being created as a post-cluster-creation step where all the control-plane and worker nodes are being labelled together with the same function/logic. Hence, kubectl seemed the cleaner and more straightforward approach to implement the label creation.

That's my perspective @BenTheElder , what do you think

[BenTheElder]

If we go with kubelet's way of applying node labels, then, we'd have to code different separate logics for adding labels to worker nodes (using kubelet at the time of registering nodes) and adding labels to control-plane nodes (using kubectl or something else).

Huh? All of the nodes run kubelet and are registered. The labels are attached to the node at registration time by kubelet. The control-plane kubelet registers as well ...

Also, here, in the code, the labels are being created as a post-cluster-creation step where all the control-plane and worker nodes are being labelled together with the same function/logic. Hence, kubectl seemed the cleaner and more straightforward approach to implement the label creation.

kind is not a general command runner and we already have a (bad) abstraction for various stages of doing things.

[yashvardhan-kukreja]

If we go with kubelet's way of applying node labels, then, we'd have to code different separate logics for adding labels to worker nodes (using kubelet at the time of registering nodes) and adding labels to control-plane nodes (using kubectl or something else).

Huh? All of the nodes run kubelet and are registered. The labels are attached to the node at registration time by kubelet. The control-plane kubelet registers as well ...

Also, here, in the code, the labels are being created as a post-cluster-creation step where all the control-plane and worker nodes are being labelled together with the same function/logic. Hence, kubectl seemed the cleaner and more straightforward approach to implement the label creation.

kind is not a general command runner and we already have a (bad) abstraction for various stages of doing things.

Ohh alright. My bad. Will hop into implementing with kubelet.

[yashvardhan-kukreja]

@BenTheElder , just a small thought.

The cluster creation is happening through kubeadm in the code. And that's why , the code is not executing any kubelet commands directly while creating the cluster. (kubeadm cli is doing that, behind the scenes).
But the way you suggested; "adding labels to nodes at the time of node creation itself via kubelet" (using the kubelet --node-labels the way I see it). That would require the code to execute kubelet commands instead of/alongside kubeadm commands.

Wouldn't that be messy or at least rough to implement?

Just a thought. Please correct me if I am wrong.

[yashvardhan-kukreja]

Also, I don't know if there's anyway to do labelling of nodes via kubeadm but still, say, if there's a way to do so. Then, we would have to code the labelling of nodes at two places. One in the kubeadminit.go action (for dealing with the labelling of control-plane nodes). and other in the kubeadmjoin.go action (for dealing with the labelling of worker nodes.)

But in my approach in the current PR, it's all happening in one place in the postcreationconfiguration.go which seems cleaner and more straightforward.
Any thoughts regarding that?

[yashvardhan-kukreja]

I hope that makes sense.
(please correct me if you think I am getting wrong somewhere)

[BenTheElder]

Also, I don't know if there's anyway to do labelling of nodes via kubeadm but still, say, if there's a way to do so. Then, we would have to code the labelling of nodes at two places. One in the kubeadminit.go action (for dealing with the labelling of control-plane nodes). and other in the kubeadmjoin.go action (for dealing with the labelling of worker nodes.)

No, it would be done via the kubeadm config, by adjusting kubelet options.

[yashvardhan-kukreja]

@BenTheElder I removed all that 'postcreationconfiguration' stuff and coded the addition of labels at the time of node registration via kubeadm's kubeletExtraArgs.

[yashvardhan-kukreja]

And I tested it with the following configuration and it's working perfectly fine:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  labels:
    name: sample-controller
    age: 2
- role: control-plane
  labels:
    name: sample-controller-2
    index: 3
- role: worker
  labels:
    name: sample-worker
    worker-age: 3
- role: worker
- role: worker

Output

► kubectl get nodes --show-labels
NAME                  STATUS   ROLES    AGE   VERSION   LABELS
kind-control-plane    Ready    master   25m   v1.19.3   age=2,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-control-plane,kubernetes.io/os=linux,name=sample-controller,node-role.kubernetes.io/master=
kind-control-plane2   Ready    master   25m   v1.19.3   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,index=3,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-control-plane2,kubernetes.io/os=linux,name=sample-controller-2,node-role.kubernetes.io/master=
kind-worker           Ready    <none>   23m   v1.19.3   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-worker,kubernetes.io/os=linux
kind-worker2          Ready    <none>   23m   v1.19.3   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-worker2,kubernetes.io/os=linux,name=sample-worker,worker-age=3
kind-worker3          Ready    <none>   23m   v1.19.3   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-worker3,kubernetes.io/os=linux

[yashvardhan-kukreja]

@BenTheElder @neolit123 @krzyzacy Please review this PR

[BenTheElder]

sorry all time is currently consumed on bug-fixes (or non kind related work, none of the maintainers work full time on kind), with some small bits of time granted to documentation (which helps save everyone's time).

features and releases are on hold, in particular #1969 needs to be resolved.

I will get to this ASAP, but I need to prioritize these still.

EDIT: previously I was personally unavailable for a bit for any work for personal reasons, since I've been back I've been resolving those.

[yashvardhan-kukreja]

features and releases are on hold, in particular #1969 needs to be resolved.

Regarding this, this has already received an lgtm from Lubomir :)

sorry all time is currently consumed on bug-fixes (or non kind related work, none of the maintainers work full time on kind), with some small bits of time granted to documentation (which helps save everyone's time).

features and releases are on hold, in particular #1969 needs to be resolved.

I will get to this ASAP, but I need to prioritize these still.

EDIT: previously I was personally unavailable for a bit for any work for personal reasons, since I've been back I've been resolving those.

Sure @BenTheElder . No worries. Please take your time.
Thanks a lot for the heads up though 😄

[BenTheElder]

#2023

[BenTheElder]

sorry for the incredibly long delay. I am closing out our backlog prior to #2120 and since the last release we've expanded the maintainer group to help avoid this.

I'd still like to move this to v1alpha5, but as that doesn't exist yet, merging as v1alpha4. if we get v1alpha5 ready in time for 1.21, we'll migrate this over. if not we'll just leave it as a v1alpha4 addition.

thank you for working on this.

[k8s-ci-robot]

@yashvardhan-kukreja: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-kind-e2e-kubernetes	4b5c1bb	link	/test pull-kind-e2e-kubernetes

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[wallrj]

Some node labels in the config file (such as node-role.kubernetes.io/worker) cause kind create cluster to fail with an unfriendly error
See:

• Some node labels in the config file (such as node-role.kubernetes.io/worker) cause kind create cluster to fail with an unfriendly error #3536