Dynamic Provisioning with Topology Awareness Support

[leakingtapan]

Currently, AWS cloud interface used by EBS CSI controller service is instantiated with the same AZ as where controller service is running. This causes the issue that EBS volume is only created at the AZ where controller service runs.

For example, if there are three nodes, each in us-west-2a, us-west-2b and us-west-2c. And controller service runs in us-west-2b while application is deployed to us-west-2a. During volume creation, the new volume will always be created in us-west-2b which causes application deployment to fail (since EBS volume and instance have to be in the same AZ).

One way to fix this is using node selector to always deploy application in the same AZ as controller. Or use static provision where PV/PVC created statically. However, both approach is non-ideal. They either enforces where controller service can run or invalidates the use case of dynamic provisioning.

[ddebroy]

For Kubernetes, You will want to integrate with the topology awareness framework in kubernetes-csi/external-provisioner#141 in Kube-CSI. This will make sure CreateVolumeRequest passed down has a reference to the SelectedNode chosen for a pod where a volume needs to be provisioned when WaitForFirstConsumer is set. The AZ of the SelectedNode can be used to perform the AWS Cloud API calls to make sure the volume is provisioned in the same AZ as the SelectedNode for the pod. This will ensure parity wrt topology/zone awareness the in-tree EBS driver.

[leakingtapan]

@ddebroy thanks for the input. Will take a look at topology awareness framework.

[leakingtapan]

Researched how to leverage topology awareness for volume provisioning. Since we can safely assume EBS has only one topology key which is zone, the work requires for this feature will be:

• return topology in NodeGetInfo response eg. accessible_topology = {"zone": "us-west-2a"}
• consume topology requirement in CreateVolume request and creates volume using the given zone

Working on implementing this.