Support watching for custom values of kubernetes.io/ingress.class

[yuvipanda]

Do you want to request a feature or report a bug?

Feature

What did you do?

I have multiple traefik servers installed on a kubernetes cluster using Ingress objects as backend. Since this is running on Google Cloud, there is also a default Kubernetes Ingress provider. Currently I can specify that an Ingress object should only be processed by traefik with the kubernetes.io/ingress.class annotation to traefik. This is good - however, not good enough. I'd like to be able to customize the ingress.class of each traefik server to be something different, so they can be more specifically targeted.

Output of traefik version: (What version of Traefik are you using?)

1.3.8

What is your environment & configuration (arguments, toml, provider, platform, ...)?

     # traefik.toml
      logLevel = "INFO"
      defaultEntryPoints = ["http"]
      [entryPoints]
        [entryPoints.http]
        address = ":80"
        compress = true
      [kubernetes]

[yuvipanda]

I'm going to try to work on a small patch for this

[timoreimann]

We usually recommend to use the labelSelector Traefik feature and / or watch separate namespaces to partition responsibilities of multiple Traefik instances. I'd be interested to know if you have considered that and why it didn't work for you.

[yuvipanda]

We use the labelSelector too, but unfortunately that is not enough. Let me try to provide more details about our cluster setup and why we need this!

We have a kubernetes cluster that is set up like this:

1. Running on GKE, which by default has the google cloud ingress backend enabled
2. Since we do not want to use the default ingress provider, we run traefik in a namespace with a LoadBalancer service pointing to it.

This works great - we can hand out access to different namespaces in the cluster to different users, and they can get working ingresses by adding the kubernetes.io/ingress.class=traefik annotation.

The problem is when users want to run software that itself uses traefik + ingresses for its own purposes. In this case, we're trying to run JupyterHub, which does multi-tenant jupyter notebooks. It requires a dynamic proxy that the JupyterHub process can control (more docs at https://jupyterhub.readthedocs.io/en/latest/reference/proxy.html). We've one that is based on feeding kubernetes ingresses to a specially configured traefik (see PR here: jupyterhub/kubespawner#64).

So the JupyterHub process creates ingress objects with the expectation that it is consumed by a specially configured traefik proxy that is in the same namespace. This is impossible to achieve right now with just labels, since the cluster wide traefik proxy will still be processing all these ingresses! There's no way to instruct it to not process these ingresses asking all users of the cluster to specify a new label that then quickly becomes non-standard & specific to this cluster, which we would like to avoid.

If this patch gets accepted, when a JupyterHub sets up a traefik proxy, it can set a custom value for IngressClass, and hence better control which ingresses are processed by which proxy.

Hope that made sense! Am happy to expand more / answer questions as needed!

And thank you for making a wonderful product we can build on! <3

[timoreimann]

@yuvipanda thanks for elaborating. Things are clearer to me now -- in a nutshell, the problem you have with the label selector is that it needs to be specified in order to opt-out of Ingress objects, whereas you'd prefer opt-in.

Let's continue on the design and implementation in the PR.

[yuvipanda]

Yup, that is correct! Thank you for understanding, @timoreimann!