Redirect HTTPS port to HTTP port because of how AWS works with LoadBalancer services

[kytart]

Issue

I'm trying to setup this ingress controller on AWS. When I set it up with the service of type LoadBalancer, AWS automatically creates an AWS load balancer mapped to it's node ports, both for HTTP and HTTPS.

This works great if my SSL certificate is stored in the cluster and Nginx is using it directly to terminate SSL. However, I'd like to go a step further. AWS can store my SSL certificate and terminate SSL on the load balancer, before the communication reaches the k8s cluster. That's what I'd ike to do, because I can benefit from having my certificate managed by AWS directly.

The problem is, right now there is no way to make this work. AWS maps it's load balancer to the ports of the k8s service based on the configuration of the ports in the service. For example, if the configuration is this:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx-ingress
  name: nginx-ingress
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app: nginx-ingress

The AWS load balancer would map it's HTTP port to the HTTP node port of this service and HTTPS port to HTTPS port. If however I setup SSL termination on the AWS load balancer, it's HTTPS port is still mapped to HTTPS port of the k8s service. The result is this:

[Client] -> encrypted -> [AWS load balancer (443)] -> unencrypted -> [Nginx (443)]

As you can see, Nginx receives unencrypted traffic on port 443 which obviously doesn't work. What I want instead is the unencrypted traffic to be rerouted to port 80 of Nginx. Unfortunatelly, that can't be achieved with the current state of things.

Solution

I discussed this in this ServerFault thread. As you can see, I got a response with a suggestion, that the Nginx service should have it's ports set like this:

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-2:XXXXXXXX:certificate/XXXXXX-XXXXXXX-XXXXXXX-XXXXXXXX"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: https
      port: 443
      targetPort: http  # <--- PAY ATTENTION HERE

This works, but the current helm chart doesn't allow to configure this. It's hardcoded there:

# deployments/helm-chart/templates/controller-service.yaml
{{- if .Values.controller.service.httpsPort.enable }}
  - port: {{ .Values.controller.service.httpsPort.port }}
    targetPort: 443
    protocol: TCP
    name: https
  {{- if eq .Values.controller.service.type "NodePort" }}
    nodePort: {{ .Values.controller.service.httpsPort.nodePort }}
  {{- end }}
{{- end }}

It would be nice if there was some way of setting this. I can suggest something like

targetPort: {{ .Values.controller.service.httpsPort.targetPort }}

which is the most obvious solution. The value would default to 443 so it's backwards compatible.

[pleshakov]

@kytart Thanks for the detailed explanation! The proposed solution looks reasonable.

Would you be interested in sending a PR?

[kytart]

@pleshakov thanks, yeah I'll send PR soon!

[Rulox]

@kytart Sorry, we wanted to have this for the next release so I implemented it, thanks for the suggestion on the implementation!

[kytart]

@Rulox no problem, thanks! I was very busy and couldn't find the time. Sorry...