Load Balancer Behavior for WebSocket Protocol with Nginx

[mhsayan]

I'm planning to deploy 5 server nodes along with one Nginx as a reverse proxy. While researching, I learned that each connection involves two sockets: one between the client and Nginx, and another between Nginx and the server. Considering a scenario where each server node can handle up to 10,000 socket connections, I anticipate that a single node server can manage 10,000 socket connections, and with 5 nodes, the total capacity reaches 50,000 connections. Similarly, the Nginx reverse proxy itself can handle up to 10,000 socket connections.

My concern arises when I think about the scenario where the total number of socket connections exceeds 10,000. What happens when this limit is reached? Will the additional connections be queued, rejected, or is there any other behavior I should be aware of? Additionally, how can I efficiently manage and scale beyond this limit to ensure optimal performance and availability?

Any insights or best practices regarding this scenario would be greatly appreciated. Thank you!

[FZambia]

Hello @mhsayan

We have https://centrifugal.dev/docs/server/infra_tuning chapter which mentions important aspect of handling many connections.

Regarding this case, you will come across several main challenges on the way to 50k connections:

1. Need to increase open file limit on every server node involved (for Nginx, for Centrifugo to be able to open many file descriptors)
2. Make sure Nginx configured to work with many connections, see https://stackoverflow.com/a/8217856/1288429 . 10k is not a limit for it.
3. You may come across with ephemeral port problem at this point - https://centrifugal.dev/docs/server/infra_tuning#ephemeral-port-exhaustion . There are several techniques to deal with it and handle millions of connections without lacking ports.

Test your setup to make sure the desired number of connections is possible. Make sure you do not hit limits on the client side when performing such tests (common mistake is to try opening many connections from a single machine - but forget that there is also open file limit, ephemeral port exhaustion).

On a more massive scale (like > 200k) per server node the possible number of connections also limited by conntrack table. Netfilter framework which is part of iptables keeps information about all connections and has limited size for this information. See how to see its limits and instructions to increase in this article.