limited scalability returning JSON due to ArrayPool contention

[nathana1]

When running an mvc json benchmark (https://github.com/aspnet/benchmarks/blob/dev/src/Benchmarks/Controllers/HomeController.cs#L22) on a 48 core machine I'm unable to get above 75% cpu with profiles showing a lot of time blocked in Rent and Return in System.Buffers.DefaultArrayPool.

The issue appears to be that the ArrayPool partitions its locks based on buffer size (

Mvc/src/Microsoft.AspNetCore.Mvc.Core/Internal/MemoryPoolHttpResponseStreamWriterFactory.cs

Line 65 in a78f77a

return new HttpResponseStreamWriter(stream, encoding, DefaultBufferSize, _bytePool, _charPool);

) and only one size of buffer is requested from the byte and char pools (

Mvc/src/Microsoft.AspNetCore.Mvc.Core/Internal/MemoryPoolHttpResponseStreamWriterFactory.cs

Line 65 in a78f77a

return new HttpResponseStreamWriter(stream, encoding, DefaultBufferSize, _bytePool, _charPool);

), so all requests will wait on one bucket.

We seem to need another level of indirection/partitioning here, not sure if MVC or the ArrayPool is the best place.

[rynowak]

/cc @KrzysztofCwalina @stephentoub @jonmiller

@nathana1 - how many RPS is this running at?

We did some investigations here in the past but didn't encounter a scenario where the contention was significant after making some changes to fix it. https://github.com/dotnet/corefx/issues/7727

[nathana1]

This is about 400k rps.

[Eilon]

Parking this in post-v1 for now. When we have more data we can figure out when exactly to take a change related to this.

[rynowak]

@nathana1 it's probably a good time to start taking a look at this now.

What kind of data do we have that points to the array pool as an issue?

Do we know for sure that we were using the implementation of ArrayPool<> with the fix for https://github.com/dotnet/corefx/issues/7727 ? Note that we did not ship the fixed version of this in RC2.

[nathana1]

This still repro's in the rtm bits. I'm working on some profiles and I'll send them to you.

[nathana1]

Hi @rynowak,

In the various profiles I'm seeing 5-10% spent in spinning in the ArrayPool and the thread time profiles show another 5% spent sleeping in those same locks.

I did a quick and dirty prototype (nathana1/corefx@b5ae2a9) where I partition the ArrayPool buckets into subbuckets and lock at that level instead, I see throughput go from 330k rps to 460k rps (and cpu saturation).

/cc @stephentoub @sajaya @sivagms

[Eilon]

@rynowak can you follow up with @davidfowl on what we wish to do here?

[stephentoub]

Is this still an issue after dotnet/coreclr#8209 ?

[rynowak]

Yeah. We should close this.