Skip to content

Latest commit

 

History

History
222 lines (159 loc) · 11.9 KB

virtual-machines-node-pools.md

File metadata and controls

222 lines (159 loc) · 11.9 KB
title description ms.topic ms.custom ms.date ms.author author
Use Virtual Machines node pools in Azure Kubernetes Services (AKS)
Learn how to add multiple Virtual Machine types of a similar family to a node pool in an AKS cluster.
how-to
devx-track-azurecli
07/26/2024
wilsondarko
wdarko1

Use Virtual Machines node pools (preview) in Azure Kubernetes Service (AKS)

In this article, you learn about the new Virtual Machines node pool type (preview) for AKS.

With Virtual Machines node pools, AKS directly manages the provisioning and bootstrapping of every single node. For Virtual Machine Scale Sets node pools, AKS manages the model of the Virtual Machine Scale Sets and uses it to achieve consistency across all nodes in the node pool. Virtual Machines node pools enable you to orchestrate your cluster with virtual machines that best fit your individual workloads.

Overview

How it works

A node pool consists of a set of virtual machines, where different virtual machine sizes are designated to support different types of workloads. These virtual machine sizes, referred to as SKUs, are categorized into different families that are optimized for specific purposes. For more information, see VM SKUs.

To enable scaling of multiple virtual machine sizes, the Virtual Machines node pool type uses a ScaleProfile that contains configurations indicating how the node pool can scale, specifically the desired list of virtual machine size and count. A ManualScaleProfile is a scale profile that specifies the desired virtual machine size and count. Only one virtual machine size is allowed in a ManualScaleProfile. You need to create a separate ManualScaleProfile for each virtual machine size in your node pool.

Note

When creating a new Virtual Machines node pool, you need at least one ManualScaleProfile in the ScaleProfile. A Virtual Machines node pool can have multiple manual scale profiles.

Advantages

Advantages of the Virtual Machines node pool type include:

  • Flexibility: Node specifications can be updated to adapt to your current workload and needs.
  • Fine-tuned control: Single node-level controls allow specifying and mixing nodes of different specs to lift restrictions from a single model and improve consistency.
  • Efficiency: You can reduce the node footprint for your cluster, simplifying your operational requirements.

Virtual Machines node pools provide a better experience for dynamic workloads and high availability requirements. Virtual Machines node pools enable you to set up multiple similar-family virtual machines in one node pool. Your workload will be automatically scheduled on the available resources that you configure.

Feature comparison

The following table highlights how Virtual Machines node pools compare with standard Scale Set node pools.

Node pool type Capabilities
Virtual Machines node pool You can add, remove, or update nodes in a node pool. Virtual machine types can be any virtual machine of the same family type (for example, D-series, A-Series, etc.).
Virtual Machine Scale Set based node pool You can add or remove nodes of the same size and type in a node pool. If you add a new virtual machine size to the cluster, you need to create a new node pool.

Limitations

  • Cluster autoscaler is currently not supported.
  • InifiniBand isn't available.
  • Windows node pools aren't supported.
  • This feature isn't available in Azure portal. Azure CLI or REST APIs must be used to perform CRUD operations or manage the pool.
  • Node pool snapshot isn't supported.
  • All VM sizes selected in a node pool need to be from a similar virtual machine family. For example, you can't mix an N-Series virtual machine type with a D-Series virtual machine type in the same node pool.
  • Virtual Machines node pools allow up to five different virtual machine sizes per node pool.

Prerequisites

  • An Azure subscription. If you don't have one, you can create a free account.
  • The Virtual Machines node pool feature is in preview. This feature is only available with API versions 2023-10-02-preview and greater, or with the Azure CLI extension versions 2.61.0 or greater.
  • If using the Azure CLI, register the aks-preview extension or update the version of existing aks-preview to minimum version 4.0.0b4.
  • The minimum minor Kubernetes release version required for this feature is release 1.26.

Install the aks-preview Azure CLI extension

[!INCLUDE preview features callout]

  1. Install the aks-preview extension using the az extension add command:

    az extension add --name aks-preview

  2. Update to the latest version of the aks-preview extension using the az extension update command.

    az extension update --name aks-preview

Register the VMsAgentPoolPreview feature flag

  1. Select the subscription where you want to enable the feature flag using the az account set command.

    az account set --subscription <subscription-name>

  2. Register the VMsAgentPoolPreview feature flag using the az feature registration create command.

    az feature registration create --namespace Microsoft.ContainerService --name VMsAgentPoolPreview

    It takes a few minutes for the status to show Registered.

  3. Verify the registration status using the az feature show command.

    az feature show --namespace "Microsoft.ContainerService" --name "VMsAgentPoolPreview"

  4. When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider using the az provider register command.

    az provider register --namespace "Microsoft.ContainerService"

Create an AKS cluster with Virtual Machines node pools

Note

Only one VM size is allowed in a scale profile, and the maximum limit is five VM scale profiles overall for a Virtual Machines node pool.

  • Create an AKS cluster with Virtual Machines node pools using the az aks create command with the --vm-set-type flag set to "VirtualMachines".

    The following example creates a cluster named myAKSCluster with a Virtual Machines node pool containing two nodes, generates SSH keys, sets the load balancer SKU to standard, and sets the Kubernetes version to 1.28.5:

    az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --vm-set-type "VirtualMachines" \
    --node-count 2 \
    --kubernetes-version 1.28.5

Add a Virtual Machines node pool to an existing cluster

  • Add a Virtual Machines node pool to an existing cluster using the az aks nodepool add command with the --vm-set-type flag set to "VirtualMachines".

    The following example adds a Virtual Machines node pool named myvmpool to the myAKSCluster cluster. The node pool has three nodes and a maximum VM SKU of Standard_D4s_v3:

    az aks nodepool add \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name myvmpool \
    --vm-set-type "VirtualMachines" \
    --vm-sizes "Standard_D4s_v3" \
    --node-count 3

Add a manual scale profile to a node pool

  • Add a manual scale profile to a node pool using the az aks nodepool manual-scale add with the --vm-sizes flag set to "Standard_D2s_v3".

    The following example adds a manual scale profile to node pool myvmpool in cluster myAKSCluster. The node pool includes two nodes with a VM SKU of Standard_D2s_v3:

    az aks nodepool manual-scale add \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name myvmpool \
    --vm-sizes "Standard_D2s_v3" \
    --node-count 2

Update an existing manual scale profile

  • Update an existing manual scale profile in a node pool using the az aks nodepool manual-scale update command with the --vm-sizes flag set to "Standard_D2s_v3".

    [!NOTE] Use the --current-vm-sizes parameter to specify the size of the existing node pool that you want to update. You can update --vm-sizes and/or --node-count. When using other tools or REST APIs, you need to pass in a full agentPoolProfiles.virtualMachinesProfile.scale field when updating the node pool scale profile.

    The following example updates a manual scale profile to the myvmpool node pool in the myAKSCluster cluster. The command updates the number of nodes to five and changes the VM SKU from Standard_D4s_v3 to Standard_D8s_v3:

    az aks nodepool manual-scale update \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name myvmpool \
    --current-vm-sizes "Standard_D4s_v3" \
    --vm-sizes "Standard_D8s_v3" \
    --node-count 5

Delete a manual scale profile

  • Delete an existing manual scale profile using the az aks nodepool manual-scale delete command.

    [!NOTE] The --current-vm-sizes parameter specifies the size of the existing node pool to be deleted. When using other tools or REST APIs to update the node pool scale profile, pass in a full agentPoolProfiles.virtualMachinesProfile.scale field.

    The following example deletes the manual scale profile for the Standard_D8s_v3 VM SKU in the myvmpool node pool.

    az aks nodepool manual-scale delete \
    --resource-group myResourceGroup \
    --cluster-name myAKSCluster \
    --name myvmpool \
    --current-vm-sizes "Standard_D8s_v3"

Next steps

In this article, you learned how to use Virtual Machines node pools in AKS. To learn more about node pools in AKS, see Create node pools.