MountVolume.SetUp failed for volume "volume2" : invalid character 's' after object key:value pair

[OmpahDev]

This is similar to but not the same as closed issue #3.

I'm trying to create 2 volumes:

      volumes:
        - name: volume
          flexVolume:
            driver: "azure/blobfuse"
            readOnly: false
            secretRef:
              name: blobfusecreds
            options:
              container: repos
              tmppath: /tmp/blobfuse
              mountoptions: "--file-cache-timeout-in-seconds=120 --use-https=true"
        - name: volume2
          flexVolume:
            driver: "azure/blobfuse"
            readOnly: false
            secretRef:
              name: blobfusecreds
            options:
              container: projects
              tmppath: /tmp/blobfuse
              mountoptions: "--file-cache-timeout-in-seconds=120 --use-https=true"

and it fails with

"MountVolume.SetUp failed for volume "volume2" : invalid character 's' after object key:value pair"

This is preventing the container from starting, so there's no additional logs I can give.

I am running on AKS with K8 version 1.11.4. Blobfuse driver is installed.

[andyzhangx]

@sshorkey can you get the blobfuse-flexvol-installer.log log file by following:
https://github.com/Azure/kubernetes-volume-drivers/tree/master/flexvolume/blobfuse#debugging-skills
and also you may try removing a mountoption with mountoptions: "--file-cache-timeout-in-seconds=120

[OmpahDev]

@andyzhangx Do you know where I can get that file? I'm on AKS and connecting using the cloud shell, I don't know where that log file is or how to get it...

[andyzhangx]

@sshorkey could you remove the mountoptions in the config and retry, that will narrow down the issue.

[andyzhangx]

you could ssh to one node with problem for diagnostics:
https://docs.microsoft.com/en-us/azure/aks/ssh

[andyzhangx]

hi @sshorkey for diagnostics, you may refer to this case: #12, it could help you narrow down the issue.

[andyzhangx]

@sshorkey thanks for reporting, mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8 would fix this issue, related PR: #13
You could use following way to update the blobfuse driver manually:
https://github.com/Azure/kubernetes-volume-drivers/blob/master/flexvolume/blobfuse/install-blobfuse-manually.md#3-install-blobfuse-flexvolume-driver-on-every-agent-node

[andyzhangx]

correct way to update the blobfuse driver on AKS:

kubectl delete ds blobfuse-flexvol-installer -n flex
kubectl apply -f https://raw.githubusercontent.com/Azure/kubernetes-volume-drivers/master/flexvolume/blobfuse/deployment/blobfuse-flexvol-installer-1.9.yaml

[andyzhangx]

Let me know if you have any question, thanks.