file.NewLocalFS creates directories with 0640 permissions

[dpkirchner]

Is this a request for help? no

What keywords did you search in NGINX Ingress controller issues before filing this one? NewLocalFS, MkdirAll

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

NGINX Ingress controller version:

The version associated with the image quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.25.0

Kubernetes version (use kubectl version):

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:44:30Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.7-gke.8", GitCommit:"7d3d6f113e933ed1b44b78dff4baf649258415e5", GitTreeState:"clean", BuildDate:"2019-06-19T16:37:16Z", GoVersion:"go1.11.5b4", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Cloud provider or hardware configuration: GCP
• OS (e.g. from /etc/os-release): Container-Optimized OS from Google VERSION_ID=73 (the node's OS)
• Kernel (e.g. uname -a): Linux gke-xxx-pool-1-125ab139-kqm6 4.14.127+ #1 SMP Tue Jun 18 18:32:10 PDT 2019 x86_64 Intel(R) Xeon(R) CPU @ 2.20GHz GenuineIntel GNU/Linux (the node's kernel)
• Install tools: ?
• Others: ?

What happened:

If you mount /etc/ingress-controller as a read-write volume (as you might do if you have a PodSecurityPolicy enforcing read-only root filesystems) the ingress controller will create subdirectories ssl and auth. However, the mode on those newly created directories will be 2640, and thus not usable by a regular user.

What you expected to happen:

I expected the pod to be created with regular ol' writable directories with permissions 0750 or similar.

How to reproduce it (as minimally and precisely as possible):

The easiest method would be to create an empty helm chart, add the stable/nginx-ingress v1.9.1 helm chart as a dependency, create a values.yaml file with these values:

nginx-ingress:
  controller:
    service:
      loadBalancerIP: "127.0.0.2"
    extraVolumeMounts:
      - name: etc-ingress-controller
        mountPath: /etc/ingress-controller
    extraVolumes:
      - name: etc-ingress-controller
        emptyDir: {}

(I am not 100% sure about the load balancer IP, you may need to pick a valid one.)

Then, extract the chart tarball in the charts directory, open controller-deployment.yaml, comment out all the liveness and readiness probe values, and then replace all the args in controller-deployment.yaml with just command and args (I've included context to make it easier to find in the file):

          lifecycle:
{{ toYaml .Values.controller.lifecycle | indent 12 }}
          {{- end }}
          command: ["/bin/sh"]
          args: ["-c", "sleep 3600"]
          # args:
          #   - /nginx-ingress-controller

Now, install the chart, wait for a controller pod to start, bash in to it with kubectl exec -it podname bash, and run ls -l /etc/ingress-controller . You'll see this output:

drwxrwsrwx 5 root     daemon   4096 Aug  9 23:12 .
drwxr-xr-x 1 www-data www-data 4096 Aug  9 23:11 ..
drw-r-S--- 2 www-data daemon   4096 Aug  9 23:12 auth
drw-r-S--- 2 www-data daemon   4096 Aug  9 23:12 ssl

Anything else we need to know:

I believe the issue is here:

ingress-nginx/internal/file/filesystem.go

Line 40 in 29c5d77

func NewLocalFS() (Filesystem, error) {

although I don't know why the mode on the created directories is 2640 and not 2660 (which you'd think it would be when using ReadWriteByUser in a directory with a setgid bit set.