Support endpointslices - Prefer Same zone traffic to save costs

[szuecs]

Right now in kubernetes dataclient we use endpoints to get lb pool members. These are flat ip port tuples.

In Kubernetes there is a new feature endpointslices that have topology which enables us to have endpoints for specific zones for example. With this we can optimize traffic costs, because same zone traffic costs less than cross zone traffic.
See also KEP topology per endpoint and KEP API

In a discussion in slack a user wrote:

for us, the cost savings are huge. We've already implemented it in a few places in the application and it's saved about 5% of our total AWS bill.

5% is huge, and I think we to have have more priority for this

see also:

• https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/
• https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/

Steps

• integrate endpointsclies Use endpointslices instead of endpoints  #2551
• zone aware routing endpointslices support zone aware routing #2552

[szuecs]

Interesting KEP updates hints
https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/2433-topology-aware-hints

[szuecs]

See also https://twitter.com/a_sykim/status/1413492899258400770?s=21 for changes in kubernetes 1.22

[szuecs]

kubernetes/enhancements#4003 is an interesting change proposal within kubernetes

[szuecs]

A quick hack to enable endpointslices usage would be to have an annotation on ingress or routegroup to force service use instead of endpoints https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/

[AlexanderYastrebov]

A quick hack to enable endpointslices usage would be to have an annotation on ingress or routegroup to force service use instead of endpoints

I think we have a flag for that introduced by #2088

[szuecs]

@AlexanderYastrebov yes but by annotation would reduce the blast radius to test. If you have a cluster with a few pods and hug number of pods side-by-side than the few pods can easily be impaired.

Example you have a backend with 8 pods, 3 in zone A and B and 2 in C.
Now skipper-ingress has 4 pods, 1 in zone A and B and 2 in C.
NLB and other internal clients will spread the load across all 4 skipper pods and now skipper has 50% traffic in C and only 25% in A and B. Now let's assume we use zone aware routing in skipper, then you will get 50% traffic to C but you have 8 pods, so you serve 50% traffic with 25% of your pods. Would you think it's a great solution?
I don't think it is, but this would be the way how it works.