Conflicting comment and implementation for kubelet_enforce_node_allocatable

[yuha0]

The current sample group_vars has this comment telling users to use comma to separate levels in kubelet_enforce_node_allocatable:

A comma separated list of levels of node allocatable enforcement to be enforced by kubelet.
Acceptable options are 'pods', 'system-reserved', 'kube-reserved' and ''. Default is "".

kubespray/inventory/sample/group_vars/k8s_cluster/k8s-cluster.yml

Lines 257 to 259 in 09748e8

	# A comma separated list of levels of node allocatable enforcement to be enforced by kubelet.
	# Acceptable options are 'pods', 'system-reserved', 'kube-reserved' and ''. Default is "".
	# kubelet_enforce_node_allocatable: pods

However, in kubelet config template, the value is split by .split()

kubespray/roles/kubernetes/node/templates/kubelet-config.v1beta1.yaml.j2

Lines 18 to 24 in 09748e8

	{% if kubelet_enforce_node_allocatable is defined and kubelet_enforce_node_allocatable != "\"\"" %}
	{% set kubelet_enforce_node_allocatable_list = kubelet_enforce_node_allocatable.split() %}
	enforceNodeAllocatable:
	{% for item in kubelet_enforce_node_allocatable_list %}
	- {{ item }}
	{% endfor %}
	{% endif %}

split() uses whitespace as delimiter by default, not comma.

If I write this in my group_vars:

kubelet_enforce_node_allocatable: pods,system-reserved

I will get this in the rendered kubelet config file:

enforceNodeAllocatable: 
- pods,system-reserved

And kubelet will refuse to start:

Failed to validate kubelet configuration" err="invalid configuration: option \"pods,system-reserved\" specified for enforceNodeAllocatable

IMHO, we should either

1. change the template to use comma as delimiter (split(',')), or
2. Change the comment section to tell users to use whitespace as delimiter

It's worth noting that, even after fixing this particular issue,kubelet_enforce_node_allocatable still cannot take anything other than pods. This is because the current kubelet config template does not have systemReservedCgroup or kubeReservedCgroup fields and they are not configurable. These two fields are required for adding kubeReserved/systemReserved to enforceNodeAllocatable. See documentation for kubelet configuration:

https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/
enforceNodeAllocatable
[]string

This flag specifies the various Node Allocatable enforcements that Kubelet needs to perform. This flag accepts a list of options. Acceptable options are none, pods, system-reserved and kube-reserved. If none is specified, no other options may be specified. When system-reserved is in the list, systemReservedCgroup must be specified. When kube-reserved is in the list, kubeReservedCgroup must be specified.

It seems like the two fields can be added via #9209

[yuha0]

Looks like this issue was brought up again in #9693 and fixed in #9694. Thanks @Tristan971 !

Closing...

[Tristan971]

Oh I’d completely missed this issue somehow when reporting it!

Well, all’s good that ends well :-)