nginx.ingress.kubernetes.io/auth-url Can't use IP whitelisting because the auth-service recieves an internal IP

[martonsz]

FEATURE REQUEST

NGINX Ingress controller version: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.17.1

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:13:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.5-gke.4", GitCommit:"6265b9797fc8680c8395abeab12c1e3bad14069a", GitTreeState:"clean", BuildDate:"2018-08-04T03:47:40Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Cloud provider or hardware configuration: Google kubernetes engine
• OS (e.g. from /etc/os-release): Container-Optimized OS - 10452.101.0
• Kernel (e.g. uname -a): Linux gke-k8s-gnp-070a1811-63kp 4.14.22+ #1 SMP Wed May 30 13:49:27 PDT 2018 x86_64 Intel(R) Xeon(R) CPU @ 2.50GHz GenuineIntel GNU/Linux
• Install tools: Helm chart for nginx ingress v 0.25.0
• Others:

What:
I tried to create an IP whitelist in my basic authentication service. Which is a small python Flask app. But I noticed that the service receives an internal IP address when I visit an URL I want to protect with basic auth.
I tried to to use the header X-forward-for in my authentication service. But it also contained an internal IP.

This is how I annotate my ingress objects.

nginx.ingress.kubernetes.io/auth-url: "https://basicauth.mydomain.com/"

[aledbf]

@martonsz please make sure you are using --set controller.publishService.enabled=true in the installation of the chart

[martonsz]

@aledbf I added the variable but it did not help. I am still getting local IPs

The --publish-service argument is now there for the container when I reinstall the chart.

$ kubectl get deployment ingress-nginx-ingress-controller -o yaml
...
     app: nginx-ingress
        component: controller
        release: ingress
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --default-backend-service=default/ingress-nginx-ingress-default-backend
        - --publish-service=default/ingress-nginx-ingress-controller
        - --election-id=ingress-controller-leader
        - --ingress-class=nginx
...

Here are some logs from my basic auth service showing the request headers.

This one is when visiting the address https://admin.mydomain.com/links which is annotated with nginx.ingress.kubernetes.io/auth-url: "https://basicauth.mydomain.com/". Here I receive a local IP 10.12.0.1 which is not one of my pods.

X-Request-Id: f4ecdae649b7fd875313ca902323c08e
X-Real-Ip: 10.12.0.1
X-Forwarded-For: 10.12.0.1
X-Forwarded-Host: basicauth.mydomain.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Original-Uri: /
X-Scheme: https
X-Original-Url: https://admin.mydomain.com/links
X-Original-Method: GET
X-Sent-From: nginx-ingress-controller
X-Auth-Request-Redirect: /links
Cache-Control: max-age=0
Authorization: Basic YWRtaW46YWRtaW4=
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Wind....

When I visit the basic auth service directly using its adress https://basicauth.mydomain.com I get my public IP.

Host: basicauth.mydomain.com
Connection: close
X-Request-Id: 3997649e47666019710010629177fdca
X-Real-Ip: 212.32.136.42
X-Forwarded-For: 212.32.136.42
X-Forwarded-Host: basicauth.mydomain.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Original-Uri: /
X-Scheme: https
Cache-Control: max-age=0
Authorization: Basic YWRtaW46YWRtaW4=
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Wi...

[aledbf]

@martonsz please check https://github.com/kubernetes/ingress-nginx/blob/master/deploy/provider/cloud-generic.yaml#L9 is present

[martonsz]

@aledbf Yes. I am using externalTrafficPolicy: Local. That is why I get my public IP when visiting the basicauth.mydomain.com directly ( as shown in the second log example).

[aledbf]

@martonsz this is a bug. We should send the client IP address in the header X-Forwarded-For