-
Notifications
You must be signed in to change notification settings - Fork 848
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to configure load balancing of HTTP2? #1726
Comments
So you have only one destination url (what you pass to the forwarder), but that destination itself has load-balancing? This is essentially the issue discussed in #1555:
Is this just an observation when testing, or do you need load balancing on a more granular level? |
Hi @MihaZupan, I definitely want to reuse the Http2 connection for multiple requests but in a way so that I keep a Pool of opened connections based on DNS record and then always round-robin requests between those connections. This is basically client-based load balancing. Docs for implementation of this in GRPC client is here: |
Your best option right now is to do the DNS lookup before IHttpForwarder and put the IP directly in the destination url. That way you can manually round robin between IPs. |
@Tratcher I expected this to already be implemented behind IHttpForwarder as an option since in GRPC Dotnet library this is not a few lines of code, it's very hard logic with a lot of knowledge behind it. Also, I see that YARP users would benefit having this feature. |
There is no built-in DNS round-robin in .NET after Framework. This is the issue discussing adding support for it: dotnet/runtime#68967 Even with that, doing per-request round-robin over a list of HTTP/2 connections is not something HttpClient supports. If there is sufficient load to warrant having multiple H2 connections open, it will still use connections to their allowed maximum. |
I'm not sure I'd want to build this directly into SocketsHttpHandler, but could be created as a wrapper layer around it/HttpClient:
|
Dns round-robin is definitely something you can do. I have an implementation here that effectively does what you describe and plugs into What I think does belong inside |
Hi @MihaZupan, Thanks for sharing your implementation. I will give it a try. I hope we will have a built-in mechanism in the near future. |
@MihaZupan I tried your NuGet package but I getting log:
Only on the first call, and after that it is kept to use only a single IP address from DNS results UPDATE I understood from your example that I need to create SockerHttpHandler per each request? Will this mean that every time a new port for connection is opened? I think this will lead to the issue of opening too many ports since OS needs time to release the port that was used. |
Enabling DnsRoundRobin means that if we end up opening multiple connections, you would see that those go to different IPs1. The second part of the problem is whether we will open multiple connections or not. By default, So you would see the desired behavior if:
For nr. 3, there is currently no way to force the handler to be more aggressive in opening connections if it doesn't think it needs them.
No, the handler should be created once and reused. Footnotes
|
@MihaZupan I misunderstood it then. If I have to generate "enough load" then I don't need a DNS load balancer then. |
We implemented gRPC load balancing @JamesNK might be able to share some experience here. |
Triage: It is worth revisiting the BCL feature for |
gRPC load balancing resolves an address (e.g. Relying on |
Hey @vasicvuk I think the problem you are having is because of k8s as described here gRPC Load Balancing on Kubernetes without Tears. We solved it by forwarding every outgoing gRPC call through an Envoy sidecar (The article suggests Linkerd though). If I understand correctly, if we want to solve this problem using YARP (HttpClient) we would need to integrate with the k8s API? |
The obvious way to do this is with the k8s API, this is being worked on as part of #1254. |
Some details
I tried to find in documentation how should we configure the Round-robin Load balancing of HTTP2. We are using IHttpForwarder but each time the connection is established all the requests are going to the one instance of the service which has multiple instances.
Since we use Kubernetes, we noticed that on the GRPC client for .NET there is the possibility to set a DNS-based Round-Robin load balancer based on Kubernetes Headless service.
Can I somehow configure this using YARP, I tried searching the documentation but I didn't find anything about it?
How many backends are in your application?
How do you host your application?
The text was updated successfully, but these errors were encountered: