ServicePointManager settings for WebJobs

Configuring ServicePointManager for WebJobs

Please note, the following guidance is based on running the WebJobs SDK on .NET Framework and might not be accurate when running on .NET Core.

The .NET Framework contains an API called ServicePointManager.DefaultConnectionLimit that is used to control the number of concurrent connections to a specific host. This setting can be confusing, but in the context of WebJobs, it is recommended that you increase this value from the default of 2 before starting your WebJobs host.

The exact value that you set this to can vary based on the design of your WebJobs, but even setting the value to Int32.MaxValue should work in the majority of cases, and is the default value configured by ASP.NET applications. This value will likely be acceptable for any WebJobs running in a Basic or above App Service Plan, which is required for the Always On setting.

If your WebJob is running in a Free or Shared App Service Plan, your application will be restricted by the App Service Sandbox which currently has a connection limit of 300. With an unbound connection limit, it's more likely that connection threshold and the site will be shut down. In that case, setting DefaultConnectionLimit to something lower, like 50 or 100, can prevent this from happening and still allow for sufficient throughput.

It is important that this setting is configured before any HTTP requests are made. For this reason, it can be unreliable for the WebJobs host to adjust the setting automatically; there may be HTTP requests that occur before the host starts and this can lead to unexpected behavior. The best approach is to set the value immediately in your Main method, before initializing the JobHost:

static void Main(string[] args)
{
    // Set this immediately so that it is used by all requests.
    ServicePointManager.DefaultConnectionLimit = Int32.MaxValue;

    var host = new JobHost();
    host.RunAndBlock();
}

Notes

The key to this API that it controls the connections to a specific host. Setting this value to 50 doesn't mean that you'll have only 50 outgoing connections maximum. It means that you'll have 50 connections maximum to each host. If you're connecting to multiple hosts, that number can increase without any limit (unless limited the operating system or sandbox).

For example, if have DefaultConnectionLimit set to 50 and you make 200 requests to http://examplesite1.com, you will immediately make the first 50 requests, and the next 150 will be queued up waiting to be sent.

If you immediately send 200 requests to http://examplesite2.com at the same time as those to http://examplesite1.com, those connections will be controlled by a separate queue. So you'll have 100 outgoing requests -- 50 to each host. And all other requests will be queued.

The client sending the requests (HttpClient in most cases) doesn't know about this queueing, so it's possible that it times out before the server even sees the requests. Or it can report a much larger latency from the client than the server sees. If the server processes requests in 50ms, but you're seeing multiple-second latencies from the client-side, it's possible that this setting is your bottleneck.