Releases: Azure/AKS
Release 2024-10-25
Release 2024-10-25
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20241025
.
Announcements
- AKS version 1.28 End of Life is Jan, 15 2025.
- AKS will be upgrading the KEDA addon to more recent KEDA versions. The AKS team has added KEDA 2.15 on AKS clusters with K8s versions >=1.32, KEDA 2.14 for Kubernetes v1.30 and v1.31. KEDA 2.15 and KEDA 2.14 will introduce multiple breaking changes. View the troubleshooting guide to learn how to mitigate these breaking changes.
- AKS will no longer support the GPU image (preview) to provision GPU-enabled AKS nodes. Starting on Jan 10, 2025 you will no longer be able to create new GPU-enabled node pools with the GPU image. Alternative options that are supported today and recommended by AKS include the default experience with manual NVIDIA device plugin installation or the NVIDIA GPU Operator, detailed in AKS GPU node pool documentation.
- Starting on January 1, 2025, invalid values sent to the Azure AKS API for the properties.mode field of AKS AgentPools will be rejected. Prior to this change, unknown modes were assumed to be User. The only valid values for this field are the (case-sensitive) strings:"User", "System", or "Gateway".
- AKS will start to block new cluster creation with basic load balancer in January 2025. Basic Load Balancer will be deprecated September 31 2025 and affected clusters must be migrated to the Standard Load Balancer prior to that date. Refer to BLB deprecation announcement for more information.
- As of November 30th, 2024, new AKS clusters created with Kubernetes versions 1.28 and 1.29 will no longer enable beta Kubernetes APIs. This matches the behavior of AKS 1.27 LTS and AKS 1.30+ clusters, which no longer enable beta APIs.
Release Notes
-
Features:
- AKS patch versions 1.28.14, 1.29.9, 1.30.5 are now available. Refer to version support policy and upgrading a cluster for more information.
- AKS version
1.31
is now generally available. Please check the release tracker for when your region will receive the GA update. Some regions may not receive this update until later in November. - The first official patch version of AKS LTS 1.27, 1.27.100, is being released.
- GitHub Copilot for Azure now supports AKS commands.
- You can now skip one release while upgrading Azure Service Mesh as long as the destination release is a supported revision - for example, asm-1-21 can upgrade directly to asm-1-23.
- You can now fine-tune supported models on KAITO version 0.3.1 with the AI toolchain operator add-on on your AKS cluster.
-
Preview features:
- We've added a new way to optimize your upgrade process drain behavior. By default, a node drain failure causes the upgrade operation to fail, leaving the undrained nodes in a schedulable state, this behavior is called
Schedule
. Alternatively, you can select theCordon
behavior, which skips nodes that fail to drain by placing them in a quarantined state, labeling themkubernetes.azure.com/upgrade-status:Quarantined
and proceeds with upgrading the remaining nodes. This ensures that all nodes are either upgraded or quarantined. This approach allows you to troubleshoot drain failures and gracefully manage the quarantined nodes. - You can now block pod access to the Azure Instance Metadata Service (IMDS) endpoint to enhance security.
- Azure Linux v3 is now in preview for AKS 1.31 clusters. After registering the preview flag
AzureLinuxV3Preview
newly created AzureLinux node pools will receive the v3 image. Existing Azure Linux v2 node pools will not upgrade to v3 and must be recreated to upgrade.- NOTE: Azure Linux v3 changes the cryptographic provider to OpenSSL + SymCrypt. The SymCrypt library will operate in FIPS mode but is still in the final stages of the validation process and thus is not considered to be FIPS-validated at this time. Do not use this preview with FIPS-enabled node pools if you must use a FIPS-validated cryptographic library.
- We've added a new way to optimize your upgrade process drain behavior. By default, a node drain failure causes the upgrade operation to fail, leaving the undrained nodes in a schedulable state, this behavior is called
-
Behavior change:
- Virtual Machine node pools creation will be blocked if the cluster is using system-assigned identity and bring-your-own virtual network, as this combination does not function properly. To utilize virtual machine node pools, migrate the cluster to a user-assigned managed identity with the required permissions on the virtual network. Virtual Machine Scale Set pools are unaffected by this change.
- Enabling long term support no longer changes the default cluster upgrade channel to
patch
. - AKS CoreDNS configuration will now block all queries ending in
reddog.microsoft.com
and some queries ending ininternal.cloudapp.net
from being forwarded to upstream DNS when they are the result of improper search domain completion. See the documentation for more details. - Azure NPM's CPU request has been lowered from 250m to 50m.
- Azure CNI Overlay now checks that the pod CIDR does not conflict with any subnet in the virtual network, rather than checking if it conflicts with the virtual network address space as a whole.
-
Component updates:
- gMSA support is updated to version v0.10.0, adding support for random hostnames and fixing an issue with multiple containers invalidating domain trusts.
- Image Cleaner has been upgraded to v1.4.0-1.
- The following Azure CSI drivers have been updated:
- Azure Blob CSI Driver: v1.22.9 for AKS 1.27, v1.23.9 for AKS 1.28 and 1.29, and v1.24.5 for AKS 1.30+
- starting from v1.23.9 and v1.24.5, blobfuse mount would respect
http_proxy
,https_proxy
environment variables
- starting from v1.23.9 and v1.24.5, blobfuse mount would respect
- Azure Disk CSI Driver: v1.28.11 for AKS 1.27, v1.19.10 for AKS 1.28 and 1.29, and v1.30.5 for AKS 1.30+
- Azure Files CSI Driver: v1.28.13 for AKS 1.27, v1.29.9 for AKS 1.28, v1.30.6 for AKS 1.29+
- Azure Blob CSI Driver: v1.22.9 for AKS 1.27, v1.23.9 for AKS 1.28 and 1.29, and v1.24.5 for AKS 1.30+
- Azure Monitor for Containers has been upgraded to 3.1.24.
- AKS Windows Server 2019 image has been updated to AKSWindows-2019-17763.6414.241010.
- AKS Windows Server 2022 image has been updated to AKSWindows-20348.2762.241009.
- AKS Azure Linux image has been updated to 202410.27.0.
- AKS Ubuntu image has been updated to 202410.27.0.
- cost-analysis-agent image has been updated to v0.0.18
- ip-masq-agent image has been updated to v0.1.14
- Components in the AKS run-command image have been added and upgraded
- New components: jq, awk, grep, xargs
- Upgraded: kubectl to v1.30.5, helm to 3.15.4
Release 2024-10-06
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20241006
.
Announcements
- AKS version 1.30 is now available as a Long term support version and AKS version 1.28 End of Life is Jan, 15 2025.
- Upgrade from LTS 1.27 to LTS 1.30 is now supported.
- AKS will be upgrading the KEDA addon to more recent KEDA versions. The AKS team has added KEDA 2.15 on AKS clusters with K8s versions >=1.31, KEDA 2.14 for Kubernetes v1.30. KEDA 2.15 and KEDA 2.14 will introduce multiple breaking changes. View the troubleshooting guide to learn how to mitigate these breaking changes.
- AKS will no longer support the GPU image (preview) to provision GPU-enabled AKS nodes. Starting on Jan 10, 2025 you will no longer be able to create new GPU-enabled node pools with the GPU image. Alternative options that are supported today and recommended by AKS include the default experience with manual NVIDIA device plugin installation or the NVIDIA GPU Operator, detailed in AKS GPU node pool documentation.
- Starting on January 1, 2025, invalid values sent to the Azure AKS API for the properties.mode field of AKS AgentPools will be rejected. Prior to this change, unknown modes were assumed to be User. The only valid values for this field are the (case-sensitive) strings:"User", "System", or "Gateway".
Release Notes
-
Preview features:
- AKS version
1.31
is now available in preview. - You can now specify the GPU driver type when creating a new AKS Windows GPU Nodepool using the
--driver-type
flag. - You can now assign a static egress gateway node pool to provide a stable egress IP for your pods.
- AKS version
-
Bug fixes:
- Bug fix to address an issue where Calico pods were stuck in Terminating state.
- Fixed a race condition in Azure Network Policy when editing or deleting then re-adding a network policy without a CIDR handle.
- Fixed a race condition between Cilium and Retina CRDs for Cilium (when Retina is updating to Cilium).
- Bug fix for certificate rotation in the gMSA webhook.
- Bug fix for Advanced Network Observability where the Retina operator didn't have proper permissions.
- Bug fix to address an issue where the Retina operator was not reading the configuration from the ConfigMap.
-
Behavior change:
- Deprecated API detection will now only show usage on non-readonly verbs (ie: not GET/LIST/WATCH).
- Starting with AKS version 1.31, nodes will now pull container images in a parallel by default. In versions prior to 1.31, the pull type will remain serialized.
- When cloud-node-manager-windows enables Windows HostProcess containers, a Windows DaemonSet will be deployed to initialize kube-proxy.
-
Component updates:
- Updated CNI and CNS versions to
v1.6.7
. - Updated Azure Network Policy Manager (NPM) to
v1.5.37
. - Updated Azure Policy addon to
v1.7.1
. - Updated konnectivity-agent image version to
v0.30.3-hotfix.20240819
. - Updated containerd-spin-shim to
v0.15.1
. - Updated Istio-based service mesh add-on revision
asm-1-23
to patchv1.23.1
.asm-1-20
is now unsupported. Users can restart the workload pods to trigger re-injection of the newer patch version of istio-proxy. More information can be found here. - Updated Cilium to
v1.14.15-241002
. - Updated Calico to
v3.28.1
. - Updated ama-logs to
v3.1.24
. - Updated azure-cloud-controller-manager to versions
v1.31.1
,v1.30.7
,v1.29.11
,v1.28.13
. - Updated overlay-vpa to
v1.2.1
for Kubernetes 1.31.0+ andv1.0.0
for Kubernetes 1.27.0+. - Azure Linux image has been updated to
Azure Linux-202403.25.0
. - Azure Linux image has been updated to
Azure Linux-202409.30.0
. - AKS Ubuntu 22.04 image has been updated to
AKSUbuntu-202409.30.0
.
- Updated CNI and CNS versions to
Release 2024-09-18
Release 2024-09-18
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20240918.
Announcements
- AKS version 1.30 is now available as a Long term support version and AKS version 1.28 End of Life is Jan, 15 2025.
- AKS will be upgrading the KEDA addon to more recent KEDA versions. The AKS team has added KEDA 2.15 on AKS clusters with K8s versions >=1.31, KEDA 2.14 for Kubernetes v1.30. KEDA 2.15 and KEDA 2.14 will introduce multiple breaking changes which are listed below:
- KEDA 2.15 for Kubernetes >=1.31: The removal of Pod Identity support. If you use pod identity, we recommend you move over to workload identity for your authentication.
- KEDA 2.14 for Kubernetes = 1.30: The removal of Azure Data Explorer 'metadata.clientSecret' as it was not safe for managing secrets.
- KEDA 2.14 for Kubernetes = 1.30: Removal of the deprecated metricName from trigger metadata section. The two impacted Azure Scalers are Azure Blob Scaler and Azure Log Analytics Scaler. If you are using
metricName
today, please movemetricName
outside of trigger metadata section totrigger.name
in the trigger section to optionally name your trigger. To view an example of what this would look like, please view the open GitHub issue.
- AKS will no longer support the GPU image (preview) to provision GPU-enabled AKS nodes. Starting on Jan 10, 2025 you will no longer be able to create new GPU-enabled node pools with the GPU image. Alternative options that are supported today and recommended by AKS include the default experience with manual NVIDIA device plugin installation or the NVIDIA GPU Operator, detailed in AKS GPU node pool documentation.
- Starting on January 1, 2025, invalid values sent to the Azure AKS API for the properties.mode field of AKS AgentPools will be rejected. Prior to this change, unknown modes were assumed to be User. The only valid values for this field are the (case-sensitive) strings:"User", "System", or "Gateway".
Release Notes
-
Features:
- AKS patch versions 1.28.13, 1.29.8, 1.30.4 are now available. Refer to version support policy and upgrading a cluster for more information.
-
Bug fixes:
- Bug fix to address the issue where the OSDiskSize validator throws an error if the existing agent pool does not have a default value set
- Bug fix causing cluster creation to fail when creating a new cluster with multiple agent pools using the Dynamic Pod IP Allocation feature (podsubnet)
- Resolved a race condition that could occur when deleting a CNI Overlay cluster with auto-scaler enabled, ensuring smoother cluster deletion.
-
Behavior change:
- Abandoned cluster will be deallocated with status
Failed(Deallocated)
instead ofSucceeded (Stopped)
. - PDB drain errors will now include additional PDB debug message and appropriate original error instead of generic "API call to Kubernetes API Server failed" error message. Example - "PDB debug info: myNode/myPod1 blocked by pdb myPDB (MaxUnavailable: 1) with 1 unready pods: myNode/myPod2".
- Updated Azure NPM version to v1.5.36 to address race condition in Azure NPM Linux which can occur when editing/deleting a NetworkPolicy with "enough" rules. The race can result in unexpected connectivity for traffic to/from Pods on the impacted Node. NPM will now auto-restart to mitigate the issue ~15 seconds after if it enters a broken state caused by the race.
- Lowering Linux Azure NPM's CPU request from 250m to 50m. This addresses [Github Issue 2792](#2792.
*Clusters using the Key Management Service (KMS) plugin based on Azure Key Vault with a private endpoint and konnectivity tunnel may run into a deadlock issue resulting inapiserver
becoming unreachable. Clusters using this configuration will not be allowed starting Kubernetes version >= 1.31. - Allow Istio add-on users to add the customizations to the Ingress gateway.
- Busybox will be removed from kube-proxy init container. This will eliminate the need for security updates on busybox.
- Abandoned cluster will be deallocated with status
-
Component updates:
- All revisions of Azure Service Mesh use zipkin as the default tracer config.
- Cost-analysis-agent image upgraded from v0.0.16 to v0.0.17.
- Updated retina linux to v0.0.15.
- Updated ip-masq-agent to v0.1.13 to address CVE-2024-24790, CVE-2023-45288, CVE-2023-45289, CVE-2023-45290, CVE-2024-24783, CVE-2024-24784, CVE-2024-24785, CVE-2024-24789, CVE-2024-24791, CVE-2024-5321.
- Updated CNI versions to v1.5.35 and v1.6.5. Updated CNS versions to v1.5.35 and v1.6.5.
- Updated Azure Container Instances (ACI) connector addon to v1.6.2 and init-validation to v0.3.0.
- Azure Monitor managed service for Prometheus images updated to 09-16-2024 release.
- Updated Azure Disk CSI driver version to v1.29.9 on AKS 1.28, 1.29, and to v1.30.4 on AKS 1.30.
- Updated Azure File CSI driver to v1.29.8 on AKS 1.28.
- Updated tigera operator to v1.30.11 and calico to v3.26.5 for versions running on k8s 1.29 and 1.30 to address CVE patches.
- Updated the Advanced Container Networking Services Image tag for fixing the bug that causes cilium pods to crash in Advanced Container Networking Service enabled AKS clusters.
- Retina Enterprise and Operator image update [v0.1.0].
- Updated the Windows containerd version from v1.6.21 to v1.6.35 for Kubernetes version < 1.28.
- AKS Windows Server 2022 image has been updated to AKSWindows-2022-20348.2700.240911.
- AKS Windows Server 2019 image has been updated to AKSWindows-2019-17763.6293.240911.
- Azure Linux image has been updated to Azure Linux-202409.09.0.
- AKS Ubuntu 22.04 image has been updated to AKSUbuntu-202409.09.0.
Release 2024-08-27
Release 2024-08-27
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20240827.
Announcements
- AKS version 1.27 is now deprecated. Enable long-term support for AKS versions if you still need to operate on 1.27.
- The attestation report for CIS Kubernetes V1.9.0 Benchmark is published which covers AKS 1.27.x through AKS 1.29.x.
- AKS will be upgrading the KEDA addon to more recent KEDA versions. The AKS team has added KEDA 2.15 on AKS clusters with K8s versions >=1.31, KEDA 2.14 for Kubernetes v1.30. KEDA 2.15 and KEDA 2.14 will introduce multiple breaking changes which are listed below:
- KEDA 2.15 for Kubernetes >=1.31: The removal of Pod Identity support. If you use pod identity, we recommend you move over to workload identity for your authentication.
- KEDA 2.14 for Kubernetes = 1.30: The removal of Azure Data Explorer 'metadata.clientSecret' as it was not safe for managing secrets.
- KEDA 2.14 for Kubernetes = 1.30: Removal of the deprecated metricName from trigger metadata section. The two impacted Azure Scalers are Azure Blob Scaler and Azure Log Analytics Scaler. If you are using
metricName
today, please movemetricName
outside of trigger metadata section totrigger.name
in the trigger section to optionally name your trigger. To view an example of what this would look like, please view the open GitHub issue.
Release Notes
-
Features:
- Existing Linux node pools can now be updated to enable or disable Federal Information Process Standard (FIPS). See documentation for more information.
-
Bug fixes:
- Fix an Azure NPM issue that user could meet unexpected connectivity for Pods on the Node when editing a NetworkPolicy with a CIDR "except" field.
- Fix bug to block non-VMSS (VirtualMachineScaleSets) agent pools in the Automatic SKU validation process.
- Fix bug to ensure correct default network plugin settings for Kubernetes clusters using VMAS.
- Fix bug for intermittent precondition failures when applying an AKS Bicep deployment on the pod subnet delegation.
- Fix bug of public IP on VMSS dropped after upgrade node image or reset service principal operation.
- Fix bug #4282 to remove duplicated toleration from Calico components.
- Fix bug to ensure
AnnotationControlled
is correctly populated by default when creating AKS clusters with app routing enabled, and to ensureAnnotationControlled
is an accepted value for the default nginx ingress controller config for AKS clusters with K8s versions <1.30. - Fix bug for Cluster Autoscaler that requires an implementation of the
HasInstance
method on AKS. This implementation prevents the Cluster Autoscaler from stalling during scale-up due to node scale-down issues. - Fix bug Azure/azure-service-operator#3220 to allow creation of AgentPools without
Count
field specified if autoscaler enabled. - Fix bug to accept user to set the
PowerState
field for API versions that do not support the filed. Impacted API versions are 2020-09-01, 2020-11-01, 2020-12-01, 2021-02-01 and 2021-03-01.
-
Behavior change:
- For non-host network pods running on AKS nodes, they cannot access wireserver(168.63.129.16) port 32526. Before this change user cannot access wireserver port 80, but port 32526 is accessible.
- When deploying an AKS Automatic (preview) cluster, user do not need to register extra feature flags for related preview features, such as APIServerVnetIntegration, NRGLockdown, NodeAutoProvisioning, and Safeguards.
- CBL-Mariner 1.0 is end of life, creation of new nodepools with OSSKU cblmariner is disabled.
- Application Gateway Ingress Controller addon has been assigned the network contributor role.
-
Component updates:
- AKS Ubuntu 22.04 image has been updated to AKSUbuntu-202408.27.0.
- Azure Linux image has been updated to AzureLinux-202408.27.0.
- Azure Disk CSI driver has been upgraded to v1.30.3 on AKS 1.30, V1.29.8 on AKS 1.28, 1.28.1 on AKS 1.27.
- Azure Blob Disk CSI driver has been upgraded to v1.24.3 on AKS 1.30, v1.23.7 on AKS 1.29 and 1.28.
- Azure File CSI driver has been upgraded to v1.30.5 on AKS 1.30 and 1.29, v1.29.7 on AKS 1.28.
- AKS Windows Server 2019 image has been updated to AKSWindows-2019-17763.6189.240814.
- AKS Windows Server 2022 image has been updated to AKSWindows-2022-20348.2655.240814.
- AKS App Routing operator image has been updated to v0.2.3-patch-2 for AKS cluster with K8s versions >=1.30, v0.2.1-patch-4 for AKS cluster with K8s versions <1.30 to address CVEs.
- Windows containerd has been updated to v1.7.20 in AKS cluster with K8s versions >= v1.28.
- Kubernetes Secrets Store CSI Driver has been updated to v1.4.4 and Azure Key Vault Provider for Secrets Store CSI Driver to v1.5.3
- Application Gateway Ingress Controller add-on image has been updated to v1.7.5.
- Retina Enterprise and Operator image has been updated to v0.0.9.
- azure-cloud-controller-manager has been updated to version v1.30.5, v1.29.9, v1.28.11, v1.27.19.
- KEDA addon has been updated to v2.14.1 for Kubernetes = 1.30.
- Azure Policy addon has been updated to v1.7.0.
- Istio-based service mesh add-on revision asm-1-20 has been upgraded to patch v1.20.8, revision asm-1-21 has been upgraded to patch v1.21.5, and revision asm-1-22 has been upgraded to patch v1.22.3. Users can restart the workload pods to trigger re-injection of the newer patch version of istio-proxy. More information can be found here.
- Calico v3.28.1 is supported for AKS cluster with K8s versions 1.31.
Release 2024-08-05
Release 2024-08-05
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20240805.
Announcements
- AKS will be upgrading the KEDA addon to more recent KEDA versions. The AKS team has added KEDA 2.15 on AKS clusters with K8s versions >=1.31, KEDA 2.14 for Kubernetes v1.30. KEDA 2.15 and KEDA 2.14 will introduce multiple breaking changes which are listed below:
- KEDA 2.15 for Kubernetes >=1.31: The removal of Pod Identity support. If you use pod identity, we recommend you move over to workload identity for your authentication.
- KEDA 2.14 for Kubernetes = 1.30: The removal of Azure Data Explorer 'metadata.clientSecret' as it was not safe for managing secrets.
- KEDA 2.14 for Kubernetes = 1.30: Removal of the deprecated metricName from trigger metadata section. The two impacted Azure Scalers are Azure Blob Scaler and Azure Log Analytics Scaler. If you are using
metricName
today, please movemetricName
outside of trigger metadata section totrigger.name
in the trigger section to optionally name your trigger. To view an example of what this would look like, please view the open GitHub issue.
Release Notes
-
Features:
- AKS version 1.30 is now available and will be the next LTS version of AKS. You can now upgrade your 1.27 clusters to 1.30 during the LTS period.
- Updating an existing node pool to enable or disable FIPS is now Generally Available.
- AKS patch versions 1.30.3, 1.29.7, 1.28.12, 1.27.16 are now available. Refer to version support policy and upgrading a cluster for more information.
- Istio add-on now only allows
EnvoyFilter
s of the types Lua, local rate limiting, and gzip compression. - Telemetry API v1 is now available for the Istio based service mesh add-on.
- The AKS extension for Visual Studio Code now supports the ability to attach an ACR to your cluster, generate Kubernetes deployment files, generate Dockerfiles, and generate GitHub Actions.
- The ignore-daemonsets-utilization, daemonset-eviction-for-empty-nodes, and daemonset-eviction-for-occupied-nodes parameters on the cluster autoscaler profile are GA from API version 2024-05-01 onwards. If you are using the CLI to update these flags, please ensure you are using version 2.63 or later.
-
Bug fixes:
- Fixed a bug where sometimes
NodePublicIPPrefixID
could show unset on a cluster even though it was set. - Previously, as part of Istio addon canary upgrade, users had to manually copy their edits to HorizontalPodAutoscaler from old revision to new revision. This has been fixed so that changes done to Horizontal Pod Autoscaler will be automatically copied for the newer revision.
- Added validation that if a LTS cluster has a node pool on non-LTS version, upgrade to the next LTS version is blocked.
- Fixed a bug where sometimes
-
Behavior change:
- When Advanced Networking Observability is enabled, increased memory limit of 700Mi (from 400Mi) is used for retina-agent.
GOMAXPROCS
for coredns has been set to equal CPU limit to avoid throttling.- In Azure CNI,
init-cni-dropgz
initContainer has been renamed tocni-installer
. - Validation for minimum 5 minutes has been introduced for drain timeout value to prevent drain issues during upgrade.
query
label removed fromdns
metrics in Advanced Network Observability.- Control plane only AKS upgrades will now reconcile node pools to desired state. For example, previously let's say a user did did a Kubernetes upgrade and network plugin mode transition to overlay where a reimaging of the nodes was required, but it wasn't done as nodes were skipped. Going ahead nodes will be reconciled in these circumstances.
-
Component updates:
- To address scheduler issues fixed in this upstream change, 1.27.15, 1.28.11, 1.29.6 schedulers versions will be used for Kubernetes versions 1.27.14, 1.28.10, 1.29.5 respectively.
- Updated Azure Blob CSI driver to v1.22.7 on AKS version 1.27.
- For Node Auto Provisioning, Azure provider of Karpenter is upgraded to v0.5.1.
- Updated Azure Monitor Container Insights image to v3.1.23.
- Azure Monitor managed service for Prometheus images updated to 07-19-2024 release.
- Updated Eraser version to v1.3.1 for Image Cleaner.
- Updated Azure Disk CSI driver to v1.28.9 on AKS 1.27 and to v1.29.7 on AKS 1.28 and 1.29.
- Updated Azure File CSI driver to v1.28.11 on AKS 1.27, to v1.29.6 on AKS 1.28, and to v1.30.3 on AKS 1.29.
- Updated Ratify image used in Image Integrity to v1.2.0.
- Updated Cilium version has been updated to 1.14.12 for AKS cluster with versions >= 1.29 and Advanced Network Observability enabled.
- Istio-based service mesh add-on revision asm-1-21 has been upgraded to patch v1.21.4 and revision asm-1-22 has been upgraded to patch v1.22.2. Users can restart the workload pods to trigger re-injection of the newer patch version of istio-proxy. More information can be found here.
- Updated Windows Kubernetes packages in all AKS versions to address CVE-2024-5321.
- AKS Ubuntu 22.04 image has been updated to AKSUbuntu-202407.29.0.
- Azure Linux image has been updated to AzureLinux-202407.29.0.
- AKS Windows Server 2019 image has been updated to AKSWindows-2019-17763.6054.240716.
- AKS Windows Server 2022 image has been updated to AKSWindows-2022-20348.2582.240716.
Release 2024-07-16
Release 2024-07-16
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20240716.
Announcements
- 1.30 is the next LTS version after 1.27. Upgrade from 1.27 LTS to 1.30 LTS will be possible starting August 2024. More information about AKS LTS is available here.
- 1.27 community supported has ended. To exit Long-Term Support (LTS) upgrade to 1.28 and disable LTS with the
az aks upgrade
command. - A new tutorial has been released on how to securely scale your applications using the KEDA add-on and workload identity.
- Running
az aks get-versions
now returns all patch versions for an AKS version. Each supported minor version can support any number of patches at a given time. AKS reserves the right to deprecate patches if a critical CVE or security vulnerability is detected. To learn more about AKS versioning, please read the official documentation. - Currently 1.25 + clusters which are running Microsoft Defender for Containers are running v1, and 1.29 + clusters which are running Microsoft Defender for Containers are running v2. Starting in mid-September 2024 all 1.25 + clusters which are running Microsoft Defender for Containers v1 will be migrated to Microsoft Defender for Containers v2.
Release Notes
-
Features:
- AKS version 1.30 is now GA.
- AKS patch versions 1.30.2, 1.30.1, 1.29.6, 1.28.11, 1.27.15, are now available. Refer to version support policy and upgrading a cluster for more information.
- Istio add-on for AKS now supports EnvoyFilter of the type Lua
(type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua)
. While thisEnvoyFilter
is allowed, any issue arising from the Lua script itself is not supported. OtherEnvoyFilter
types currently remain blocked. - The ability to migrate your existing Ubuntu node pools to Azure Linux by changing the OS SKU of the node pool is now GA.
-
Preview features:
- CNI Overlay dual-stack (IPv4/IPv6) is now available on Windows Agent Pools.
- Existing node pools can now be updated to enable or disable Federal Information Process Standard (FIPS). See aka.ms/aks/updatefips for more information.
-
Bug Fixes:
- Updated iptables rules in clusters with Azure Network Policy Manager to block pod access to wireserver.
- A bug regarding App Routing's placeholderPod not properly cleaning up has been fixed.
- A regression in kube-scheduler impacting Kubernetes versions 1.27.14, 1.28.10, 1.29.5 has been fixed in new patch versions 1.30.2, 1.30.1, 1.29.6, 1.28.11, 1.27.15. If you are on the affected patch versions. Please follow release v20240716 and upgrade your AKS version once the release is in your region.
-
Behavior Change:
- AKS Automatic clusters now use Azure Linux for Node Auto Provision dynamic nodes.
-
Component Updates:
- AKS has released new patches v1.29.5 and v1.30.4 for Cluster Autoscaler to fix a bug which prevents scaling from zero of selected SKUs.
- coreDNS has been updated to use image v1.9.4-hotfix.20240704 to fix CVE vulnerabilities.
- Istio add-on has been bumped to include v1.21.3-hotfix.20240626 and v1.22.1-hotfix.20240626 to fix datadog tracer zero-day crash.
- KEDA add-on has been updated to v2.14.0 on AKS v1.30.
- AKS Ubuntu 22.04 image has been updated to AKSUbuntu-202407.08.0.
- Azure Linux image has been updated to AzureLinux-202407.08.0.
Release 2024-06-27
Release 2024-06-27
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20240627.
Announcements
- Starting 1.30 Kubernetes version and 1.27 LTS versions, beta APIs will be disabled by default, when you upgrade to them. There will be an option provided to explicitly enable beta APIs closer to the 1.30 release.
- 1.30 is the next LTS version after 1.27. Upgrade from 1.27 LTS to 1.30 LTS will be possible starting August 2024. More information about AKS LTS is available here.
Release Notes
-
Features:
- AKS patch versions 1.27.14, 1.28.10, and 1.29.5, are now available. 1.27.9, 1.28.5, and 1.29.2 patch versions are deprecated. Refer to version support policy and upgrading a cluster for more information.
- Cost Analysis views for AKS are now available under AKS resource blade in Azure portal. More information can be found in this document.
-
Preview feature:
- Windows Server Annual Channel for Containers is now in public preview on Azure Kubernetes Service (AKS). More information can be found here.
-
Bug Fixes:
- Fixed a bug that previously didn't allow switching from non-LTS K8s version to LTS K8s version when upgrading the cluster. For example, you can now upgrade from 1.26 to 1.27 while switching to LTS.
- Related to the above, also fixed a bug where previously it was not possible to upgrade from an LTS K8s version to non-LTS K8s version. For example, you can now upgrade from 1.27 LTS to 1.28.
-
Behavior Change
- The memory limit for Azure Key Vault provider for Secrets Store CSI Driver has been updated from 300Mi to 500Mi.
- Base CPU and memory for metrics-server container are updated from 44M to 150M and 51Mi to 100Mi respectively on clusters with K8s version >= 1.30.0. More information on metrics server scaling can be found here.
- Creation of clusters with konnectivity and private Key Management Service (KMS) plugin based encryption of etcd using Azure Key Vault is no longer supported. Only clusters with API Server VNet Integration (preview) tunnel are allowed to be used along with KMS encrypted etcd clusters based on private Azure Key Vault.
-
Component Updates:
- Linux Network Policy Manager has been upgraded from v1.5.23 to v.1.5.29 to address CVE-2024-28085.
- Upgraded Azure workload identity to v1.3.0.
- Upgraded ip-masq-agent-v2 to v0.1.11 having fixes for CVE-2024-2961 and CVE-2024-33599.
- Upgraded Azure Monitor Container Insights image to v3.1.22
- Upgraded Azure CNS to v1.6.0 for 1.30 version clusters.
- Istio-based service mesh add-on revision asm-1-19 has been upgraded to patch v1.19.10-hotfix.20240528, asm-1-20 has been upgraded to patch v1.20.7, and asm-1-21 has been upgraded to patch v1.21.3. These contain fixes for CVE-2024-34362, CVE-2024-32974, CVE-2024-32975, CVE-2024-34363, CVE-2024-34364, CVE-2024-32976, CVE-2024-23326.
Users can restart the workload pods to trigger re-injection of the newer patch version of istio-proxy. More information can be found here. - AKS Ubuntu 22.04 image has been updated to AKSUbuntu-202406.19.0.
- Azure Linux image has been updated to AzureLinux-202406.19.0.
- AKS Windows Server 2022 image has been updated to AKSWindows-2022-20348.2529.240621.
Release 2024-06-09
Monitor the release status by regions at AKS-Release-Tracker. This release is titled as v20240609.
Announcements
- Starting 1.30 Kubernetes version and 1.27 LTS versions, beta apis will be disabled by default, when you upgrade to them. There will be an option provided to explicitly enable beta apis closer to the 1.30 release.
- Starting 1.30 Kubernetes version, apiserver will have
--service-account-extend-token-expiration
set to false on OIDC issuer enabled clusters. In versions prior to 1.30, service account tokens injected into pods were given an extended lifetime so they remain valid even after a new refreshed token is provided. Prior to upgrading to 1.30, the metric serviceaccount_stale_tokens_total and the audit annotation authentication.k8s.io/stale-token can be used to monitor for workloads that depend on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container. If no action is taken, workloads depending on the extended lifetime will break once the cluster is upgraded to 1.30. See reference for details. - Istio service mesh addon revision asm-1-19 is no longer supported. If you are still using this revision on your cluster, please upgrade for continued support. More information about mesh upgrades and version support can be found here.
- Container Insights has automatically migrated from legacy authentication to managed authentication on AKS clusters where the Container Insights addon was enabled with legacy authentication. This migration occurs when any feature, such as the cost-analysis addon or authorized IP ranges, is enabled using the preview API version 2023-07-02-preview or later. This unintended migration has caused monitoring to break, this issue has been fixed for new clusters. To mitigate this issue on existing clusters, re-onboarding or re-configuring of Container Insights is required.
Release Notes
-
Features:
- Generally Available - Security Patch channel - Live patching mechanism for VHD updates.
- AKS Patch version 1.27.13 is now available.
-
Preview Features:
- AKS version 1.30 is available in preview.
-
Bug Fixes:
- CoreDNS has been updated to use image v1.9.4-hotfix.20240520 on all AKS clusters above version 1.24. This updated image addresses CVE vulnerabilities.
- Updated cilium to version 1.14.10 for K8s version 1.29+, to fix the issue where the host network is broken and remains broken even if the underlying interface goes up again.
- Removes the post-upgrade annotation on hubble-generate-cert Job. On each aks cluster reconcile, the helm chart revision is incremented which counts as an upgrade. Each time the helm chart is upgraded or installed this job will restart. This change fixes that to not restart on helm chart upgrades and successfully clean up.
- Windows containerd has been upgraded from v1.7.14 to v1.7.17 in K8s v1.28+. This upgrade fixes two bugs resulting in a wrong default path and a deadlock issue.
- Fixed the following issues for AKS Edge zone support -
- Fixed bug where clusters with ExtendedLocation set would accept create node pool with availability zones even though availability zones aren't supported in ExtendedLocation mode.
- Fixed bug where
edgezone
was previously being wrongly accepted in small case. OnlyEdgeZone
is accepted.
-
Component Updates:
- Changing cilium operator tolerations to match cilium-agent. Adding tolerations for NoExecute and NoSchedule. This should fix a race condition in upgrades, where cilium-operator cannot schedule due to node taint.
- Retina Enterprise and Operator image update v0.0.8.
- Updated linux cni versions to v1.4.54 and v1.5.28.
- Gatekeeper is updated to 3.16 for kubernetes versions 1.27+.
- Updated Cilium to v1.13.13 for Kubernetes v1.28.0+.
- Upgrade azure disk csi-drivers to 1.29.6 on AKS 1.28 and 1.29.
- Updated the aks app routing operator nginx version from 1.9 to 1.10.
- AKS Ubuntu 22.04 image has been updated to AKSUbuntu-202406.07.0.
- Azure Linux image has been updated to AzureLinux-202406.07.0.
- AKS Windows Server 2019 image has been updated to AKSWindows-2019-17763.5936.240612.
- AKS Windows Server 2022 image has been updated to AKSWindows-2022-20348.2527.240612.
Release 2024-05-13
Release 2024-05-13
Monitor the release status by regions at AKS-Release-Tracker.
Announcements
- Starting 1.30 Kubernetes version and 1.27 LTS versions, beta apis will be disabled by default, when you upgrade to them. There will be an option provided to explicitly enable beta apis closer to the 1.30 release.
- Introducing the AKS blog and the AKS Youtube community
- In 2020 Docker enacted a Rate Limiting policy for all users. In-order to assist customers with the change, Microsoft worked directly with Docker to prevent users of Microsoft Azure from being impacted. However, beginning on June 30th, 2024, Azure customers will begin to be impacted by this limit. In-order for customers to mitigate the potential effects of this limit. We recommend customers begin to use the Artifact Cache feature within Azure Container Registry or sign up for a Docker Subscription. More information is available here.
- GetOSOptions will no longer be included in new AKS API versions starting with 2024-05-02. This API was used to get OS options that support Federal Information Process Standard (FIPS) in the specified subscription. If you're calling this API via the CLI, it will no longer be available in newer az aks extension versions. You can use an older version of the az aks extension, however this is not recommended. The CLI preview version supporting the 2024-05-02 preview API can be found here. Check the link for the release version.
For details on what AKS supported operating systems support Federal Information Process Standard (FIPS), see aka.ms/aks/GetFIPSOSOptions.
Release Notes
-
Features:
- Generally Available - AKS supports disabling Windows OutboundNAT.
- Generally Available - Automated Deployments.
- Generally Available - Security patch channel for VHD updates.
- Generally Available - Azure Kubernetes Fleet Manager workload orchestration
- AKS Patch version 1.28.9 is now available. It fixes Bug - OpenAPI handler fails on duplicated path.
-
Preview Features
- Deployment Safeguards now supports mutations in Enforcement mode.
- Enable Native sidecars mode for Istio-based service mesh addon in AKS.
- AKS Automatic. Visit the AKS engineering blog post.
- Node Initialization Taints.
- Advanced Container Networking Services can be enabled on Cilium-enabled clusters with Kubernetes v1.29.0 or greater, and on Retina-enabled clusters with Kubernetes v1.21.0 or greater for Advanced Network Observability.
- Allow disabling NPM for existing clusters with "networkPolicy=none" for stable api version 2024-05-01.
- Property-based scheduling in Azure Kubernetes Fleet Manager.
- Cluster resource overrides in Azure Kubernetes Fleet Manager.
- Service Connector on AKS cluster. It simplifies the connection configuration experience for AKS workloads and Azure backing services such as Azure Key Vault, Storage account and Azure OpenAI.
-
Behavioral Changes:
- Node upgrade (reimage) will wait for disk detach to complete (to prevent very slow disk detach).
- Default network policy is "networkPolicy=none" when network policy is not set on new clusters starting from API version 2024-05-01.
- Customized apiserver subnet must be empty when migrating a cluster to enable apiserver-vnet-integration. If the subnet has resources in it, the migration won't be allowed.
-
Bug Fixes:
- Metrics Server v0.6.3 will be used to prevent frequent OOMKills, reverting from v0.7.1.
- Allowing zonal NodeClaims to facilitate NodeClaims and node creation on Node Auto Provisioning for workloads with zone affinity constraints.
- Fixed a bug where the SSHAccess property of a node pool would be reset to LocalUser(SSHAccess:LocalUser) on a partial put. Henceforth, SSHAccess property will retain the current value (SSHAccess:current value).
- Fixed bug where the eTag property in 2024-02-02 preview, 2024-03-02 preview, and 2024-04-02 preview APIs was returned with the wrong case (returned etag, should have been eTag).
-
Component Updates:
- Istio-based service mesh add-on revision asm-1-19 has been upgraded to patch v1.19.10, asm-1-20 has been upgraded to patch v.1.20.6, and asm-1-21 has been upgraded to patch v1.21.2. Users can restart the workload pods to trigger re-injection of the newer patch version of istio-proxy. More information can be found here.
- Linux and Windows addon-token-adapter image for Azure monitoring metrics is updated to mcr.microsoft.com/aks/msi/addon-token-adapter:master.240510.2. The updated image patches CVE-2023-4911,CVE-2024-2961, CVE-2024-33599, CVE-2024-33600, CVE-2024-33601, CVE-2024-33602, CVE-2023-3446, CVE-2023-3817, CVE-2023-3446, CVE-2023-3817.
- Managed Prometheus image version updated to images:6.8.12-main-05-21-2024.
- Azure Policy addon has been updated to v1.4.0 for all clusters on Kubernetes version >= v1.25.
- Updated cloud node manager to v1.30.0 on AKS 1.30+, v1.29.4 on AKS 1.29+, v1.28.9 on AKS 1.28+, v1.27.17 on AKS 1.27+. Refer AKS version matrix for cloud node manager.
- Updated AKS App Routing operator image to v0.2.3.
- Updated Azure File CSI driver to v1.28.10 on AKS 1.27, v1.29.5 on AKS 1.28, v1.30.2 on AKS 1.29.
- AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202405.20.0.
- Azure Linux image has been updated to AzureLinux-202405.20.0.
- AKS Windows Server 2019 image has been updated to AKSWindows-2019-17763.5820.240516.
- AKS Windows Server 2022 image has been updated to AKSWindows-2022-20348.2461.240516.
Release 2024-04-28
Monitor the release status by regions at AKS-Release-Tracker.
Announcements
- Starting 1.30 Kubernetes version and 1.27 LTS versions, beta apis will be disabled by default, when you upgrade to them. There will be an option provided to explicitly enable beta apis closer to the 1.30 release.
- On 15 March 2027, Windows Server 2022 will be retired when Kubernetes 1.34 reaches the end of platform support. You won't be able to create new Windows Server 2022 node pools on Kubernetes 1.35 and above. We encourage you to make the switch before 15 March 2027 to gain the richer benefits of Windows Server 2025 or Windows Server Annual Channel. These new Windows OS versions will be supported on AKS before Windows Server 2022 is retired. For more updates, see our AKS public roadmap.
- In 2020 Docker enacted a Rate Limiting policy for all users. In-order to assist customers with the change, Microsoft worked directly with Docker to prevent users of Microsoft Azure from being impacted. However, beginning on June 30th, 2024, Azure customers will begin to be impacted by this limit. In-order for customers to mitigate the potential effects of this limit. We recommend customers begin to use the Artifact Cache feature within Azure Container Registry or sign up for a Docker Subscription. More information is available here
- If you use any programming/scripting logic to list and select a minor version of Kubernetes before creating clusters with the
ListKubernetesVersions
API, note that starting from Kubernetes v1.27, the API returnsSupportPlan
as[KubernetesOfficial, AKSLongTermSupport]
. Please ensure you update any logic to excludeAKSLongTermSupport
versions to avoid any breaks and chooseKubernetesOfficial
support plan versions. Otherwise, if LTS is indeed your path forward please first opt-into the Premium tier and theAKSLongTermSupport
support plan versions from theListKubernetesVersions
API before creating clusters. Refer long term support for more information.
Release Notes
-
Features:
- With this release, Azure Linux 2.0 becomes a supported OS for AKS Long Term Support (LTS) with v1.27. Learn more about Azure Linux and LTS.
- You can now get insight into the progress of any ongoing operation, such as create, upgrade, and scale, using any preview API version after
2024-01-02-preview
using the Get/List operations call. Refer to Long running operations on an Azure Kubernetes Service (AKS) cluster for more information. - AKS patch version 1.29.4 is now available.
-
Behavioral Changes:
- Manually added Labels, Taints, and Annotations on nodes will no longer be copied to nodes during surged upgrade. To ensure any Label or Taint is present in new nodes please use the Labels and/or Taints functionality provided by AKS.
- The Istio-based service mesh add-on now skips validation of its compatibility with cluster version unless mesh upgrade or cluster upgrade is attempted.
- Effective starting with Kubernetes version 1.29, when you deploy Azure Kubernetes Service (AKS) clusters across multiple availability zones, AKS now utilizes zone-redundant storage (ZRS) to create managed disks within built-in storage classes. ZRS ensures synchronous replication of your Azure managed disk across multiple Azure availability zones in your chosen region. This redundancy strategy enhances the resilience of your applications and safeguards your data against datacenter failures. Refer to Storage concept for more information.
-
Bug Fixes:
- Fixed a bug that incorrectly calculated number of free IPs in a subnet when upgrading an agent pool using Azure CNI with Dynamic IP allocation.
- Fixed a bug to allow correct IP address calculation for subnets with Private Link Service.
- Fixed a bug where the ordering of the system environment variables injected into pods could change.
- Fixed a bug in clusters that use Node Autoprovisioning for stateful workloads deployments that use availability zones.
- Fixed a bug in clusters that use Node Autoprovisioning and managed identity to authenticate Azure Container Registry.
- Fixed an issue where clusters using Pod Identity would fail to migrate to Azure CNI.
- The Istio-based service mesh add-on components can now tolerate running on the system node pools with the
CriticalAddonsOnly
taint. - Fixed an issue where the ephemeral disk placement was incorrectly modified/updated on an existing nodepool.
-
Component Updates:
- Upgraded Kubernetes Secrets Store CSI Driver to v1.4.3 and Azure Key Vault Provider for Secrets Store CSI Driver to v1.5.2
- The Istio-based service mesh add-on has been patched to versions 1.19.9, 1.20.5, and 1.21.1 to address CVE-2024-27919, CVE-2024-30255, CVE-2023-45288, and GHSA-3mh5-6q8v-25wj. Workloads need to be restarted by user to consume the latest version of
istio-proxy
sidecar. More information can be found here. - Open Service Mesh (OSM) add-on has been patched to address GHSA-3mh5-6q8v-25wj.
- Upgraded Azure Disk CSI driver to v1.30.1 on AKS 1.29.
- Upgraded Azure File CSI driver to v1.29.7 on AKS 1.27 and to v1.29.4 on AKS 1.28 on Windows nodes.
- Upgraded Azure Blob CSI driver to v1.24.1 on AKS 1.28 and to v1.22.6 on AKS 1.27.
- Upgraded Azure workload identity to v1.2.2.
- Upgraded Azure Monitor Container Insights image to v3.1.20
- Windows Server 2019 has been updated to Windows Server 17763.5696.240423.
- Windows Server 2022 has been updated to Windows Server 20348.2402.240423.
- Azure Linux image has been updated to Azure Linux 202404.16.0
- AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202404.16.0