Newly introduced drivername makes csi-smb-node crash

[JesperBerggren]

What happened:
We upgraded our Helm template from v.0.7.0 to v.0.7.0 (yes, an upgrade to the same version).
After the upgrade the node, csi-smb-node, started to fail with the following error:

flag provided but not defined: -drivername
Usage of /smbplugin:
  -add_dir_header
    	If true, adds the file directory to the header of the log messages
  -alsologtostderr
    	log to standard error as well as files
  -endpoint string
    	CSI endpoint (default "unix://tmp/csi.sock")
  -kubeconfig string
    	Absolute path to the kubeconfig file. Required only when running out of cluster.
  -log_backtrace_at value
    	when logging hits line file:N, emit a stack trace
  -log_dir string
    	If non-empty, write log files in this directory
  -log_file string
    	If non-empty, use this log file
  -log_file_max_size uint
    	Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
  -logtostderr
    	log to standard error instead of files (default true)
  -metrics-address string
    	export the metrics (default "0.0.0.0:39615")
  -nodeid string
    	node id
  -skip_headers
    	If true, avoid header prefixes in the log messages
  -skip_log_headers
    	If true, avoid headers when opening log files
  -stderrthreshold value
    	logs at or above this threshold go to stderr (default 2)
  -v value
    	number for the log level verbosity
  -version
    	Print the version and exit.
  -vmodule value

We've looked into your commits and it appears that you've made a commit a couple of weeks ago that introduced this new flag.
Unfortunately it breaks the functionality - at least when using Helm.

We tried to set driver.name as a Helm argument but that did not work either.

What you expected to happen:
Absolutely nothing as we "upgraded" from the same version.

How to reproduce it:
Install the Helm chart v.0.7.0

Anything else we need to know?:
You really should make the versioned Helm charts use versioned containers rather than using the "latest" tag! We need to be sure that a versioned Helm chart will be the same on all environments.
You could simply introduce a "-devel" tag. E.g. 0.8.0-devel then users could opt-in to use a development version if needed.

Environment:

• CSI Driver version: 0.7.0
• Kubernetes version (use kubectl version): v1.21.1
• OS (e.g. from /etc/os-release): Ubuntu 18.04
• Kernel (e.g. uname -a): Linux XXX-38TWBI11 4.15.0-143-generic Fix image repo, version and flag to support ARM64 #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
• Install tools: Helm v.3
• Others:

[andyzhangx]

try run helm with --set image.smb.pullPolicy=Always to pull the image on the node, v0.7.0 is actually not released, you should use stable released version(e.g. v0.6.0) in your cluster.

Another workaround is try kubectl edit ds -n kube-system csi-smb-node to set imagePullPolicy: Always and change smb image version, to make it pull the latest image on all agent nodes.

[JesperBerggren]

Hi and thanks for quick reply!!

Unfortunately we can't revert to v.0.6.0 because you fixed another issue in v.0.7.0 so it won't help us setting the pull policy to always (because it will still just get the newest version that has the drivername error).

Is there anything we can do to help things along? It's quite urgent for us as we're in the middle of testing of our platform and solution. I guess it's some golang flag issue, but I'm no golang expert to say it the least :-)

Best regards
Jesper

[andyzhangx]

Another workaround is try kubectl edit ds -n kube-system csi-smb-node to set imagePullPolicy: Always and change smb image version(latest --> v0.7.0 or v0.7.0 --> latest), to make it pull the latest image on all agent nodes.

What issue are you mentioning?

[JesperBerggren]

Hi again,

The issue is that the latest version (both using image tag latest and 0.7.0) doesn't work when using Helm.
The csi-smb-node is in state CrashLoopBackOff because of the following error in the log (notice the line: flag provided but not defined: -drivername):

W0601 11:44:54.801089       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W0601 11:45:04.800412       1 connection.go:172] Still connecting to unix:///csi/csi.sock
I0601 11:44:45.120340       1 main.go:113] Version: v2.2.0
I0601 11:44:45.120852       1 main.go:137] Attempting to open a gRPC connection with: "/csi/csi.sock"
W0601 11:44:55.121534       1 connection.go:172] Still connecting to unix:///csi/csi.sock
W0601 11:45:05.121262       1 connection.go:172] Still connecting to unix:///csi/csi.sock
flag provided but not defined: -drivername
Usage of /smbplugin:
  -add_dir_header
        If true, adds the file directory to the header of the log messages
  -alsologtostderr
        log to standard error as well as files
  -endpoint string
        CSI endpoint (default "unix://tmp/csi.sock")
  -kubeconfig string
        Absolute path to the kubeconfig file. Required only when running out of cluster.
  -log_backtrace_at value
        when logging hits line file:N, emit a stack trace
  -log_dir string
        If non-empty, write log files in this directory
  -log_file string
        If non-empty, use this log file
  -log_file_max_size uint
        Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
  -logtostderr
        log to standard error instead of files (default true)
  -metrics-address string
        export the metrics (default "0.0.0.0:29644")
  -nodeid string
        node id
  -skip_headers
        If true, avoid header prefixes in the log messages
  -skip_log_headers
        If true, avoid headers when opening log files
  -stderrthreshold value
        logs at or above this threshold go to stderr (default 2)
  -v value
        number for the log level verbosity
  -version
        Print the version and exit.
  -vmodule value
        comma-separated list of pattern=N settings for file-filtered logging

[andyzhangx]

kubectl edit ds -n kube-system csi-smb-node

And then change image and imagePullPolicy as following, I have a cluster has same issue with yours and fix the issue by using this workaround:

      - args:
        - --v=5
        - --drivername=smb.csi.k8s.io
        - --endpoint=$(CSI_ENDPOINT)
        - --nodeid=$(KUBE_NODE_NAME)
        - --metrics-address=0.0.0.0:29645
        env:
        - name: CSI_ENDPOINT
          value: unix:///csi/csi.sock
        - name: KUBE_NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        image: mcr.microsoft.com/k8s/csi/smb-csi:v0.7.0
        imagePullPolicy: Always

[andyzhangx]

BTW, once #259 is merged, I will publish a GA release, that's a milestone.

[JesperBerggren]

Oh, you're right (of course).
Setting the imagePullPolicy to always was in fact a valid work around.
Thanks for a quick resolution to an urgent issue.

We're looking forward to the next GA release.