Update AspNetCore benchmarks

[utpilla]

Changes

• Add instrumentation options Params to have a benchmark method run for each of the possible combinations:

  • No instrumentation enabled
  • Instrumentation enabled only for Traces
  • Instrumentation enabled only for Metrics
  • Instrumentation enabled for both Traces and Metrics

• Update the web application builder setup to use async/await

• Update BenchmarkDotNet version to 0.13.5

Benchmark results:

Method	EnableInstrumentation	Mean	Error	StdDev	Gen0	Allocated
GetRequestForAspNetCoreApp	None	226.8 us	4.00 us	3.74 us	-	2.45 KB
GetRequestForAspNetCoreApp	Traces	235.2 us	4.44 us	4.15 us	0.4883	3.59 KB
GetRequestForAspNetCoreApp	Metrics	229.1 us	4.44 us	4.36 us	-	2.92 KB
GetRequestForAspNetCoreApp	Traces, Metrics	230.6 us	4.54 us	5.23 us	0.4883	3.66 KB

[codecov]

Codecov Report

Merging #4405 (c174e1b) into main (457a9e9) will decrease coverage by 0.01%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4405      +/-   ##
==========================================
- Coverage   84.75%   84.74%   -0.01%     
==========================================
  Files         300      300              
  Lines       12010    12010              
==========================================
- Hits        10179    10178       -1     
- Misses       1831     1832       +1     

see 3 files with indirect coverage changes

[cijothomas]

@utpilla is the numbers steady or fluctuating a lot? its interesting that traces+metrics takes less time than traces alone. Is it always the case, or varies?

[cijothomas]

wondering why your machine took 230 us vs 171 us in my very very old (4th gen vs 9th gen) CPU ❓

[cijothomas]

also, the overhead is now 5-8% as opposed to 10-12% before.

[utpilla]

wondering why your machine took 230 us vs 171 us in my very very old (4th gen vs 9th gen) CPU ❓

Your old machine has a higher clock speed than my new machine 😂

[utpilla]

also, the overhead is now 5-8% as opposed to 10-12% before.

I have only made two major changes:

1. The only Benchmarks tooling related difference here is that I have removed the InProcess attribute from the benchmark class. InProcess does not create a separate process for each benchmark. AFAIK, InProcess leads to a slightly better performance.
2. The other difference in this PR is in the web application builder setup which now uses async/await when configuring the route: app.MapGet("/", async context => await context.Response.WriteAsync($"Hello World!")); instead of app.MapGet("/", () => $"Hello World!");. This leads to slightly lower memory allocation across the board (uninstrumented or otherwise). I haven't investigated why but I would think it's probably causing lesser allocation for async state machines in this case.

These are the results on my machine when I run the existing benchmarks in the main repo:

BenchmarkDotNet=v0.13.3, OS=Windows 11 (10.0.23424.1000)
Intel Core i7-9700 CPU 3.00GHz, 1 CPU, 8 logical and 8 physical cores
.NET SDK=7.0.203
[Host] : .NET 7.0.5 (7.0.523.17405), X64 RyuJIT AVX2

Job=InProcess Toolchain=InProcessEmitToolchain

Method	Mean	Error	StdDev	Gen0	Allocated
UninstrumentedAspNetCoreApp	207.3 us	3.64 us	3.40 us	0.2441	2.54 KB
InstrumentedAspNetCoreAppWithDefaultOptions	210.1 us	1.93 us	1.81 us	0.4883	3.74 KB

It doesn't show a high overhead on my machine even for the existing ones 😀

[vishweshbankwar]

Do you see any notable difference if you target net6.0 instead of net7.0?

[utpilla]

My apologies! I missed this comment. I'll take a look at this as well.

[cijothomas]

It doesn't show a high overhead on my machine even for the existing ones

Yes. It simply means the benchmarks are not very steady, so can't be the sole way to measure overhead. I did see consistent numbers (showing 10+ % overhead).

[reyang]

LGTM. When will we see the AOT benchmarks?

[utpilla]

LGTM. When will we see the AOT benchmarks?

I'm not sure if Benchmarks would be needed for AOT. Benchmark dotnet does a lot of warm up iterations anyway for jitting etc. to happen before it actually starts the measurements.

We would probably need some other kind of test that would also measure the startup time of the application.