ingress won't work without NET_BIND_SERVICE cap and allow-privilege-escalation for v0.20.0 or higer

[TheKangaroo]

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG

NGINX Ingress controller version:
0.20.0-0.22.0

Kubernetes version (use kubectl version):
1.11.5

Environment:

• Cloud provider or hardware configuration: on-prem vmWare/BareMetal
• OS (e.g. from /etc/os-release): CoreOS 1745.7.0
• Kernel (e.g. uname -a): 4.14.48-coreos-r2
• Install tools: ignition/systemd units

What happened:
I try to update our nginx-ingress from 0.19.0 to 0.2x.0, but see /nginx-ingress-controller: Operation not permitted
I disallow all capabilities via PSP. this worked fine for 0.19.0 with

- --http-port=8080
- --https-port=6443

I though we only need NET_BIND_SERVICE cap and allow-privilege-escalation for the port binding to 80 and 443
This was in the release notes for 0.16.x so I wonder how this worked with 0.19.0 but doesn't work with more recent versions

What you expected to happen:
nginx-ingress still working without NET_BIND_SERVICE cap, allow privilege escalation if I configure it to listen on high ports, as it worked in 0.19.0

How to reproduce it (as minimally and precisely as possible):
Disalow privilege escalation and require to drop all capabilities.
Then run ingress-nginx with ports set to 8080 and 6443.

[mmack]

Same problem here... anyone?

[aledbf]

nginx-ingress still working without NET_BIND_SERVICE cap

That is not possible. We must be able to bind to any port for the TCP/UDP feature.

[aledbf]

Closing. Having NET_BIND_SERVICE as a requirement is something we cannot avoid to be able to run as non-root.

Edit: that said, PRs are welcome if you find a way to bind to any port and run as non-root

[TheKangaroo]

Okay, I see there is no way to run without NET_BIND_SERVICE for low ports.
I didn't think of the TCP/UDP feature.
Maybe it is possible to have a switch to turn TCP/UDP feature on and off? In this case, it would be possible to run without NET_BIND if the feature is not needed.

[aledbf]

@TheKangaroo what's the behavior if you remove NET_BIND_SERVICE cap and use a higher port?
I think this should work but is up to you to customize the deployment

[TheKangaroo]

@aledbf actually that's what I did yesterday.
I build a private pipeline with the following Dockerfile:

ARG TAG
FROM quay.io/kubernetes-ingress-controller/nginx-ingress-controller:$TAG

USER 0
RUN  setcap    cap_net_bind_service=-ep /nginx-ingress-controller \
  && setcap    cap_net_bind_service=-ep /usr/sbin/nginx
USER www-data

It works just fine, but I haven't try what will happen if I use TCP/UDP feature yet.