v1beta2 config

[rosti]

A proposal issue for the next kubeadm API version (currently proposed as v1beta2).

The actual details are in this doc.

If you are a member of the Kubernetes SIG Cluster Lifecycle Google Group you should already have edit rights for it. Ping me if you have any problems accessing the doc.

---

track PRs here:

• kubeadm: Introduce v1beta2 config kubernetes#76710
• kubeadm: Add certificateKey field to v1beta2 config kubernetes#77012
• kubeadm: Fix omitempty in v1beta2 kubernetes#77345
• kubeadm: Remove ClusterConfiguration from InitConfiguration in v1beta2 kubernetes#77739

---

• k/enhancements issue: kubeadm: graduate the kubeadm configuration enhancements#970
• KEP: kubeadm: Update the config KEP enhancements#969

/kind api-change
/assign

[yagonobre]

kubernetes/kubernetes#74216

[fabriziopandini]

@rosti there are issues asking for allowing kube-apiserver customizations at node level
#1348

[fabriziopandini]

There is also the Kube Router use case (skip kube-proxy in join/upgrade vs config)

[neolit123]

DNS local cache, but we didn't talk if we want it in the config yet.

[rosti]

@fabriziopandini we can make this more generic. We can probably add skipPhases []string to InitConfiguration and JoinConfiguration.
Seems like a nice generic solution to what you are describing here.

[fabriziopandini]

@rosti skipping phase from config can be an idea but I'm not sure it is the right solution for the problem above.
I think that "with/without kube-proxy" should be treated as a supported cluster variant, like CoreDNS/KubeDNS or internal/external etcd. That implies that the list of action should automatically adapt to this variant (instead of relying on the user picking up the right steps)

[fabriziopandini]

a request we have time to time: Improve support for Kubernetes on ARM (timeouts)

[neolit123]

a request we have time to time: Improve support for Kubernetes on ARM (timeouts)

there is a workaround by using phases and patching the api-server manifest, but also the idea to make liveness probes configurable was not accepted by Robert and Tim, when we first started seeing the problem happening. ARM is OK in general, but the problem only happens on Raspberry PI. unless someone debugs the actual problems there i don't think we should take action on the config.

[neolit123]

local etcd volumes were just requested here:
#1455

[fabriziopandini]

#1468

Etcd, controller-manager and scheduler configuration per node

[neolit123]

for reference the v1beta1 kep was here: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/0023-kubeadm-config-v1beta1.md

[ghouscht]

Are there any plans to change the hanling of extraArgs in the ControlPlaneComponent with v1beta2? The current implementation is backed by a map[string]string type and this leads to some problems we're currently facing. For example the api-server allows some arguments multiple times (--service-account-key-file and --tls-sni-cert-key). We can't handle this with the extraArgs type as it is a map which of course does not allow the same key twice and thus overrides previous declarations.

Example:

---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: 1.14.2
apiServer:
  extraArgs:
    "tls-sni-cert-key": "/etc/kubernetes/pki/apiserver.crt,/etc/kubernetes/pki/apiserver.key"
    "tls-sni-cert-key": "/etc/kubernetes/pki/mycert.crt,/etc/kubernetes/pki/mykey.key" # overrides the previous declaration

(This is probably also a problem for other control plane components but I haven't checked that.)
Or is there another way to do this which I'm currently not aware of?

[neolit123]

@ghouscht
unfortunately yes, using map[string]string does introduce this problem. we'd need something like Java's MultiValuedHashMap as a backend here.
kubeadm allows the component extra args from the CLI too, where this problem also minifests.

the workaround for now is to patch the e.g kube-apiserver.yaml manifest and the kubelet will restart it.

[neolit123]

@rosti i'm closing this as v1beta2 was added successfully.
@ghouscht please file a separate ticket that outlines the problem in #1439 (comment)
if you wish.