vSphere CPI & CSI implementation

[pierreyves-lebrun]

What type of PR is this?
/kind feature

What this PR does / why we need it:
Implement external vSphere cloud provider as per #5309.
The in-tree vSphere cloud provider has been deprecated so the out-of-tree one should now be used. Duplicate of #5861.

Tested on vSphere 6.7 U3.

Special notes for your reviewer:
In order to keep both working, I added new vSphere configuration variables which are the same as the old ones but prefixed with external_

Does this PR introduce a user-facing change?:
Users should use the new external vSphere cloud provider rather than the old in-tree one.

[k8s-ci-robot]

Hi @pierreyves-lebrun. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Miouge1]

/ok-to-test

[pierreyves-lebrun]

/assign @ant31
/assign @alijahnas

[alijahnas]

All this seems good to me. Can you just add a deployment in persistent_volumes so that we can have a storage class that works with the vsphere CSI driver? Thanks.
https://github.com/kubernetes-sigs/kubespray/tree/master/roles/kubernetes-apps/persistent_volumes
https://github.com/kubernetes-sigs/kubespray/blob/master/inventory/sample/group_vars/k8s-cluster/k8s-cluster.yml#L254

P.S. I don't understand how you got those strange commits haha

[Miouge1]

I don't have a sphere cluster available to test this on. Any ideas on how to test this with say a mock server or something?

[pierreyves-lebrun]

I don't have a sphere cluster available to test this on. Any ideas on how to test this with say a mock server or something?

Afaik, there isn't any ready-to-use solution for this. That said it is fairly easy (and free) to install ESXi and vCenter on a single host.

I remember hacking something together on a Mac mini in just one afternoon. A baremetal EC2 instance would also do the job otherwise.

[Russell-IO]

Is there any docs on how to migrate from the previous / current vsphere storage solution ?
Relevant link: https://cormachogan.com/2019/10/10/moving-a-stateful-app-from-vcp-to-csi-based-kubernetes-cluster-using-velero/

[pierreyves-lebrun]

Is there any docs on how to migrate from the previous / current vsphere storage solution ?

This still is a work in progress, I’ll be waiting for the official guide on how to perform the migration:

kubernetes/cloud-provider-vsphere#268

[Miouge1]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Miouge1, pierreyves-lebrun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Miouge1]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[asmoljo]

Hello everyone,
A few months ago I installed bare metal k8s cluster v1.18.5 (RedHat 7 servers) using Kubespray scripts. Now I am trying to configure dynamic provisioning with vSphere Storage for Kubernetes. I'm following several documents and I'm quite confused because I don't know which way to go, so I'm asking you for help. I first read this https://kubernetes.io/docs/concepts/storage/storage-classes/#vsphere and here I see that there are two providers:
CSI provisioner: csi.vsphere.vmware.com
vCP provisioner kubernetes.io/vsphere-volume
I then opened the document https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/policy-based-mgmt.html which describes the vCP provisional: kubernetes.io/vsphere-volume.
After that I found the documents:
https://github.com/kubernetes-sigs/kubespray/blob/master/docs/vsphere.md
https://github.com/kubernetes-sigs/kubespray/blob/master/docs/vsphere-csi.md
which describe the CSI provisional: csi.vsphere.vmware.com
Please can someone advise me which documents to follow if I installed the k8s cluster in the way I described above.

[pierreyves-lebrun]

Hello everyone,
A few months ago I installed bare metal k8s cluster v1.18.5 (RedHat 7 servers) using Kubespray scripts. Now I am trying to configure dynamic provisioning with vSphere Storage for Kubernetes. I'm following several documents and I'm quite confused because I don't know which way to go, so I'm asking you for help. I first read this https://kubernetes.io/docs/concepts/storage/storage-classes/#vsphere and here I see that there are two providers:
CSI provisioner: csi.vsphere.vmware.com
vCP provisioner kubernetes.io/vsphere-volume
I then opened the document https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/policy-based-mgmt.html which describes the vCP provisional: kubernetes.io/vsphere-volume.
After that I found the documents:
https://github.com/kubernetes-sigs/kubespray/blob/master/docs/vsphere.md
https://github.com/kubernetes-sigs/kubespray/blob/master/docs/vsphere-csi.md
which describe the CSI provisional: csi.vsphere.vmware.com
Please can someone advise me which documents to follow if I installed the k8s cluster in the way I described above.

The in-tree vSphere cloud provider is deprecated so please use the out-of-tree one:
https://github.com/kubernetes-sigs/kubespray/blob/master/docs/vsphere.md

It will deploy the CSI driver automatically so you don’t have to worry about it.