Scale from zero fails to work on k8s >= 1.21 Azure

[himanshu-kun]

What happened:
Scale from zero fails for Azure clusters with CSI enabled.
What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

• Create worker group with 2 zones
• keep the minimum nodes for the worker group as 1
• create a PVC referring a storage class with volumeBindingMode= Immediate and assign affinity of PV to the zone with no node (basically the one which will scale from zero)
• create a pod which uses this PVC.

You'll note that pod is not able to trigger scale-up
Anything else we need to know:
This is happening because azure csi driver places the label topology.disk.csi.azure.com/zone: <zone> on the node so that the volume can get attached to the node , but in scale from zero we are unaware that the node would have this label , and so CA thinks scaling the node won't help the volume to get attached and therefore pod can't be scheduled.
Environment:
k8s>=v1.21

[himanshu-kun]

I have verified that adding the label to the machineDeployment's nodeTemplate can solve the issue. So a change in the extension-azure would be helpful like done here
I would verify if csi drivers of other providers also use these kind of labels.

@ialidzhikov would like to listen to your thoughts as well

[himanshu-kun]

/close
The issue has been solved with gardener/gardener-extension-provider-azure#430