4.x: WebServer: provide mechanism to limit number of concurrent requests

[barchetta]

To protect against unbounded resource consumption we should consider adding mechanisms to:

• Limit the maximum number of concurrent requests, similar to Apache httpd's MaxRequestWorkers. We would need to take into account Http1 connections and Http2 streams.
• Support timeout of idle connects like Apache's TimeOut

[martinKindall]

Regarding https://openjdk.org/jeps/444, you can use Semaphores in order to limit the concurrent requests.

[tomas-langer]

There are a lot of options here:

Regarding number of requests:

1. Do we care about number of opened sockets (connections)? Quite a simple solution, same behavior regardless of HTTP/1 and HTTP/2
2. Do we care about number of concurrent tasks (virtual threads) that are created? This is more complex, as there is a big difference depending on protocol (HTTP/1 - connection, HTTP/2 - streams in parallel, WebSocket - connection) - do we want to have a global switch, or a per-protocol switch?
3. Do we care about the number of sequential tasks? (not really related, just putting it here for the full picture) - e.g. how many requests can be executed per HTTP/1 connection, how many streams can be done per HTTP/2 connection etc.

Regarding timeout of idle:

• what is the driving concern here? Is it the fear of resource starvation through a DOS attack (open as many connections as server accepts and just keep them open), or is it something else? It seems that a client keeps a connection open to have good performance on next request - so do we expect the timeout to be in minutes, hours, or seconds?

[tomas-langer]

Proposal (working on code to check if feasible):

• maxTcpConnections - maximal number of connections opened to a single listener
• maxConcurrentRequests - maximal number of requests that can be processed by a single listener

Not sure about this one, as it introduces a requirement to run a worker thread that checks this

• idleConnectionTmeout - duration that a connection can be opened and not used

[barchetta]

Your proposal looks good.

I assume "listener" is a server socket listener? So these limits would apply per port. In which case maybe they should be configurable per port. I do not think it needs to be per protocol.

I was not thinking of limits for numbers of sequential requests.

In Apache the idle timeout unit is seconds and the default looks to be 5 minutes. So by default if a connection is idle for 5 minutes it is closed.

[dalexandrov]

By the way, how about this: https://github.com/bucket4j/bucket4j