Autoscaling CAPZ Clusters

[Rotfuks]

Motivation

For cost and performance optimisation customers want their cluster sizes more dynamic. We already have autoscaling of machines inside a cluster in our vintage providers and should make sure to match that feature also in CAPZ

Todo

• conceptphase

Outcome

• CAPZ clusters automatically scale in size depending on configurable metrics and boundaries, leading to a better cost control for customers but also optionally better peak performances of the clusters overall

[Rotfuks]

We already had a first discussion on it: https://gigantic.slack.com/archives/C04887ZSU20/p1670926088947149

Conclustion:

• All infos around cluster-autoscaler in machinepools issues can be found here:
  https://github.com/giantswarm/giantswarm/issues/19313
  -- We should document all further technical infos into that ticket
• CAPZ already has a feature merged for managed machine pools:
  When creating AKS clusters using autoscaler enabled, do not make an update api call to agentpool service based on difference in node count kubernetes-sigs/cluster-api-provider-azure#2444
  -- But it is currently being refactored to match the overall CAPI approach: ✨ MachinePool annotation for externally managed autoscaler kubernetes-sigs/cluster-api#7107
• CAPZ already has an issue open for support of autoscaling on unmanaged machine pools:
  feat: respect externally managed annotation on unmanaged MachinePools kubernetes-sigs/cluster-api-provider-azure#2588

[primeroz]

Let's also remember to double check scaling to 0 support when the code support lands

[Rotfuks]

Helpful Slack Thread in the CAPZ upstream community: https://kubernetes.slack.com/archives/CEX9HENG7/p1674083817745989

[Rotfuks]

As Vintage is looking into Karpenter (https://github.com/giantswarm/flexshopper/issues/359) this might also be an interesting tool for this capability.

[Rotfuks]

Parent Duplication of #1376