Kubernetes 1.27 Major Themes

[harshanarayana]

This is a discussion to discuss Major Themes for the v1.27 release.

Here is the discussion thread from the last release cycle.

cc @kubernetes/release-team-leads
cc @AnaMMedina21 @rjsadow @sanchita-07 @yrs147

[pohly]

SIG Instrumentation: PRs are under review which convert kube-controller-manager and kube-scheduler to structured, contextual logging. I'll update this comment once I know whether that work gets completed.

The description of kubernetes/enhancements#3077 has a table where I am tracking pending PRs.

[hwinkel]

May even open tracing if you touch logging ?

[ameukam]

SIG K8S Infra / SIG Release: Freeze promotion of the container images to k8s.gcr.io.

Announcement: https://kubernetes.io/blog/2023/02/06/k8s-gcr-io-freeze-announcement/
Enhancement tracking issue: kubernetes/enhancements#3720

[saschagrunert]

SeccompDefault graduates to stable

Announcement: TBD
Enhancement: kubernetes/enhancements#2413

[saschagrunert]

Kubelet OpenTelemetry tracing graduates to beta

Announcement: TBD
Enhancement: kubernetes/enhancements#2831

cc @sallyom

[saschagrunert]

DownwardAPIHugePages graduates to stable

Announcement: TBD
Enhancement: kubernetes/enhancements#2053

[xing-yang]

Here are major themes from SIG Storage

New Features:

• The Robust VolumeManager Reconstruction feature moved to Beta in v1.27. This is a VolumeManager refactoring that allows kubelet to populate additional information about how existing volumes are mounted during the kubelet startup. In general, this makes volume cleanup more robust.
• The SELinux Relabeling with Mount Options feature moved to Beta in v1.27. This feature speeds up container startup by mounting volumes with the correct SELInux label instead of changing each file on the volumes recursively.
• The ReadWriteOncePod PV Access Mode feature moved to Beta in v1.27. This feature introduces a new ReadWriteOncePod access mode for PersistentVolumes that restricts access to a single pod on a single node.

In-tree Driver Removals (planned)

• CSI Migration for Azure Disk moved to GA in v1.24. The in-tree Azure Disk plugin is targeted for removal in v1.27.
• CSI Migration for AWS EBS moved to GA in v1.25. The in-tree AWS EBS plugin is targeted for removal in v1.27.

cc @saad-ali @msau42 @jsafrane

[aravindhp]

SIG Windows

New feature:

• NodeLogQuery has been introduce as an alpha feature. This feature helps cluster administrators debug issues with services running on nodes by allowing them to query service logs. To use the feature, ensure that the NodeLogQuery feature gate is enabled for that node, and that the kubelet configuration options enableSystemLogHandler and enableSystemLogQuery are both set to true.

[kerthcet]

From SIG Scheduling

New Features:

• Mutable Pod scheduling directives when gated: This is a new feature but starts at beta level. It's similar to mutable scheduling directives for suspended Jobs, but for Pod only. With this feature enabled, pod's node affinity and selector is mutable when blocked by a scheduling gate, which gives rights to external controllers to influence the scheduling directives. This is a derivative of Pod Scheduling Readiness.

Beta -> GA

• Mutable scheduling directives for suspended Jobs: This was introduced in v1.22 and started as a beta level, now it's stable. It allows to mutate the node scheduling directives, namely the pod template's node affinity, node selector, tolerations, schedulingGates, annotations and labels of the job's pod template, which enables a higher-level queue controller to inject such scheduling directives before un-suspending a job.

Alpha -> Beta:

• Pod Scheduling Readiness: This introduces a .spec.schedulingGates field to Pod's API, to indicate whether the Pod is allowed to be scheduled or not. External controllers can use the gates to hold a Pod from scheduling based on their business needs. The SchedulingGates plugin is enabled by default now in scheduler framework.
• Respect PodTopologySpread after rolling upgrades: This adds a new field matchLabelKeys to podTopologySpread, it only cares about the keys but values which helps to simplify the configuration in pod topology spreading, and provides a solution for unbalance scheduling in pods rolling update.

cc @ahg-g @alculquicondor @Huang-Wei