StorageClass mountOptions not being applied to mount arguments

[TiLogic]

What happened:
NFS mountOptions not applied, causing NFS mount to fail.

I1208 18:53:37.938380       1 utils.go:47] GRPC call: /csi.v1.Controller/CreateVolume
I1208 18:53:37.938406       1 utils.go:48] GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-85810e45-fc85-4dd7-8f63-a27b897104f9","parameters":{"server":"10.0.0.58","share":"/Users/user/dev/exports"},"volume_capabilities":[{"AccessType":{"Mount":{"mount_flags":["vers=3","nolock"]}},"access_mode":{"mode":5}}]}
I1208 18:53:37.941805       1 controllerserver.go:239] internally mounting 10.0.0.58:/Users/user/dev/exports at /tmp/pvc-85810e45-fc85-4dd7-8f63-a27b897104f9
E1208 18:53:48.362083       1 mount_linux.go:150] Mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t nfs 10.0.0.58:/Users/user/dev/exports /tmp/pvc-85810e45-fc85-4dd7-8f63-a27b897104f9
Output: mount.nfs: rpc.statd is not running but is required for remote locking.
mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
mount.nfs: Protocol not supported

Options: {"mount_flags":["vers=3","nolock"]}}
Output: Mounting arguments: -t nfs 10.0.0.58:/Users/user/dev/exports /tmp/pvc-85810e45-fc85-4dd7-8f63-a27b897104f9

What you expected to happen:
NFS mount options to be applied to the mount command.

How to reproduce it:
The following StorageClass and PersistentVolumeClaims were used.

# StorageClass #

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-csi
provisioner: nfs.csi.k8s.io
parameters:
  server: 10.0.0.58
  share: /Users/user/dev/exports
reclaimPolicy: Retain # only retain is supported
volumeBindingMode: Immediate
mountOptions:
  - vers=3
  - nolock

# PVC #

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: nfs-csi

Anything else we need to know?:

Environment:

• CSI Driver version:
  master
• Kubernetes version (use kubectl version):
  v1.19.4
• OS (e.g. from /etc/os-release):
  Fedora 32 (Server Edition)
• Kernel (e.g. uname -a):
  Linux 5.9.11-100.fc32.x86_64
• Install tools:
  kubectl remote install
• Others:

[andyzhangx]

already fixed in master branch, e2e logs shows mountOptions are respected in CreateVolume:

[pod/csi-nfs-controller-b798764cd-jhtkn/nfs] I1227 13:09:57.092343       1 utils.go:63] GRPC call: /csi.v1.Controller/CreateVolume
[pod/csi-nfs-controller-b798764cd-jhtkn/nfs] I1227 13:09:57.092367       1 utils.go:64] GRPC request: {"capacity_range":{"required_bytes":10737418240},"name":"pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7","parameters":{"server":"nfs-server.default.svc.cluster.local","share":"/"},"volume_capabilities":[{"AccessType":{"Mount":{"mount_flags":["hard","nfsvers=4.1"]}},"access_mode":{"mode":5}}]}
[pod/csi-nfs-controller-b798764cd-jhtkn/nfs] I1227 13:09:57.092504       1 controllerserver.go:248] internally mounting nfs-server.default.svc.cluster.local:/ at /tmp/pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7
[pod/csi-nfs-controller-b798764cd-jhtkn/nfs] I1227 13:09:57.092632       1 nodeserver.go:76] NodePublishVolume: volumeID(nfs-server.default.svc.cluster.local//pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7) source(nfs-server.default.svc.cluster.local:/) targetPath(/tmp/pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7) mountflags([hard nfsvers=4.1])
[pod/csi-nfs-controller-b798764cd-jhtkn/nfs] I1227 13:09:57.103377       1 controllerserver.go:266] internally unmounting /tmp/pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7
[pod/csi-nfs-controller-b798764cd-jhtkn/nfs] I1227 13:09:57.103548       1 nodeserver.go:119] NodeUnpublishVolume: CleanupMountPoint /tmp/pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7 on volumeID(nfs-server.default.svc.cluster.local//pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7)
[pod/csi-nfs-controller-b798764cd-jhtkn/nfs] I1227 13:09:57.123500       1 utils.go:69] GRPC response: {"volume":{"capacity_bytes":10737418240,"volume_context":{"server":"nfs-server.default.svc.cluster.local","share":"/pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7"},"volume_id":"nfs-server.default.svc.cluster.local//pvc-dbaa061f-f655-435f-8ff9-a11646c6bba7"}}

While there is issue in DeleteVolume since it cannot get mountOptions by design.

[TiLogic]

Many thanks!