Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remove interface for IResolveRequestContext #1176

Merged

Conversation

alsami
Copy link
Member

@alsami alsami commented Jul 26, 2020

Part of #1164, bringing back performance improvements

  • Rename ResolveRequestContext to DefaultResolveRequestContext
  • Rename ResolveRequestContextBase to ResolveRequestContext
  • Rename MockResolveRequestContext to ResolveRequestContextStub
  • Rename MockLifetimeScope to LifetimeScopeStub
  • Implement minimum requirements in tests for ResolveRequestContextSub

@alsami alsami changed the base branch from develop to v6 July 26, 2020 10:55
@alsami
Copy link
Member Author

alsami commented Jul 26, 2020

@alistairjevans as discussed in #1174, this brings back ResolveRequestContextBase, defining all methods as abstract.

Mind running benchmarks again?

@alsami alsami force-pushed the feat/bring_back_base_resolverequestcontext branch from d232291 to 3677371 Compare July 26, 2020 10:57
@codecov
Copy link

codecov bot commented Jul 26, 2020

Codecov Report

Merging #1176 into v6 will increase coverage by 0.05%.
The diff coverage is 88.13%.

Impacted file tree graph

@@            Coverage Diff             @@
##               v6    #1176      +/-   ##
==========================================
+ Coverage   80.18%   80.23%   +0.05%     
==========================================
  Files         187      187              
  Lines        4370     4371       +1     
  Branches      946      947       +1     
==========================================
+ Hits         3504     3507       +3     
+ Misses        415      414       -1     
+ Partials      451      450       -1     
Impacted Files Coverage Δ
...ing/Middleware/ActivatorErrorHandlingMiddleware.cs 91.30% <ø> (ø)
...Resolving/Middleware/DisposalTrackingMiddleware.cs 100.00% <ø> (ø)
...Middleware/RegistrationPipelineInvokeMiddleware.cs 100.00% <ø> (ø)
...e/Resolving/Middleware/ScopeSelectionMiddleware.cs 100.00% <ø> (ø)
...fac/Core/Resolving/Middleware/SharingMiddleware.cs 91.66% <ø> (ø)
...c/Core/Resolving/Middleware/StartableMiddleware.cs 91.66% <ø> (ø)
...Core/Resolving/ResolveRequestBeginningEventArgs.cs 0.00% <0.00%> (ø)
.../Autofac/Diagnostics/DiagnosticSourceExtensions.cs 100.00% <ø> (ø)
...Autofac/Features/Decorators/DecoratorMiddleware.cs 90.47% <ø> (ø)
src/Autofac/PipelineBuilderExtensions.cs 50.00% <ø> (ø)
... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c28403...82b643e. Read the comment docs.

@alsami
Copy link
Member Author

alsami commented Jul 26, 2020

DeepGraphResolveBenchmark Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
PR 20.40 us 0.174 us 0.136 us 0.5798 - - 19.49 KB
v6 20.47 us 0.075 us 0.070 us 0.5798 - - 19.49 KB

This is what I got on Linux, a tiny bit better. Can't test develop though.

@alistairjevans
Copy link
Member

I got the same thing basically:

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
v6 (before any refactoring) 10.54 us 0.053 us 0.047 us 2.0599 0.0458 - 16.91 KB
1174 11.50 us 0.064 us 0.057 us 2.3804 0.0610 - 19.49 KB
PR 11.22 us 0.033 us 0.029 us 2.3804 0.0610 - 19.49 KB

The red flag for me though is the fact that the original refactoring seems to have jumped the memory allocations loads...

@alsami
Copy link
Member Author

alsami commented Jul 26, 2020

It's still 0.3 us slower than v6 before refactoring as well. Maybe it's also the type casting for the interface?
The memory allocation I don't understand tbh. What has changed that should consume more memory? 🤔

@alistairjevans
Copy link
Member

alistairjevans commented Jul 26, 2020

Ok, just trying to narrow this down some more:

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
v6 pre-1173 9.603 us 0.0813 us 0.0721 us 2.0599 0.0458 - 16.91 KB
v6 post-1173 10.59 us 0.075 us 0.067 us 2.0599 0.0458 - 16.91 KB
v6 post-1174 11.23 us 0.038 us 0.032 us 2.3804 0.0610 - 19.49 KB
PR 11.03 us 0.045 us 0.040 us 2.3804 0.0610 - 19.49 KB

So, #1174 started consuming a lot more memory. Curious.

@alistairjevans
Copy link
Member

I think I might have found the memory issue; you changed the ConcurrencyMiddleware to use .Any:

image

That's going to allocate an Action instance every time we hit the loop, plus the overhead of the Any LINQ construct itself.

As a rule I'm trying to strip out any LINQ from the hot-path areas because it adds overhead.

@alsami
Copy link
Member Author

alsami commented Jul 26, 2020

Makes sense, give me a second, changing it back to not use LINQ.

@alsami
Copy link
Member Author

alsami commented Jul 26, 2020

Back to original memory-usage.

DeepGraphResolveBenchmark Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
PR 18.12 us 0.056 us 0.044 us 0.4883 - - 16.91 KB
v6 (latest) 19.78 us 0.097 us 0.091 us 0.5798 - - 19.49 KB

truly didn't know that a foreach loop is faster than a LINQ-expression.

@alistairjevans
Copy link
Member

Yeah, when you pass a lambda to a method, and that lambda captures a local variable, it allocates a new instance of an object to hold that capture+delegate. It's sneaky, but shows up under a decompiler.

I'll run those diff-ing benchmarks again tomorrow, then we can see what else (if anything) might need changing to bring that perf back up.

@alsami
Copy link
Member Author

alsami commented Jul 26, 2020

I'll run those diff-ing benchmarks again tomorrow, then we can see what else (if anything) might need changing to bring that perf back up.

Sounds good!

@alsami
Copy link
Member Author

alsami commented Jul 26, 2020

@alistairjevans sorry for cluttering the PR, please feel free to delete the comments.

If you want to run benchmarks, you can do so now automated. Takes a while though, even though my machine is rather powerful.

@autofac autofac deleted a comment from alsami Jul 27, 2020
@autofac autofac deleted a comment from autofac-bot bot Jul 27, 2020
@autofac autofac deleted a comment from alsami Jul 27, 2020
@autofac autofac deleted a comment from autofac-bot bot Jul 27, 2020
@autofac autofac deleted a comment from alsami Jul 27, 2020
@autofac autofac deleted a comment from autofac-bot bot Jul 27, 2020
@autofac autofac deleted a comment from autofac-bot bot Jul 27, 2020
@autofac autofac deleted a comment from autofac-bot bot Jul 27, 2020
@alistairjevans
Copy link
Member

Ran the benchmarks again (manually):

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
v6 pre-1173 9.486 us 0.0497 us 0.0464 us 2.0599 0.0458 - 16.91 KB
v6 post-1173 10.51 us 0.042 us 0.040 us 2.0599 0.0458 - 16.91 KB
v6 post-1174 11.31 us 0.101 us 0.094 us 2.3804 0.0610 - 19.49 KB
PR 10.50 us 0.052 us 0.049 us 2.0599 0.0458 - 16.91 KB

So, the memory issue has been fixed, and we're back to the fact that #1173 introduced the main slow down. #1173 change the resolve context to an interface, but we've just changed it back...very strange. I'll take another look at #1173.

@alsami
Copy link
Member Author

alsami commented Jul 27, 2020

If you check out the diff between #1173 and #1174, you will see that there was an increase in time as well. #1174.

I'd try to check if it's related to the interface casting. Interface casting can be a bit more costly AFAIK. The diff here is 1µs, this could be really it.

@alistairjevans
Copy link
Member

alistairjevans commented Jul 27, 2020

The 1173 to 1174 drop in perf were due to the additional memory allocation; the changes in this PR bring that perf back down to post-1173 levels.

We've now taken ResolveRequestContext back to being a class, so unless I'm missing something, something in 1173, besides the change to interfaces, caused a performance problem.

One thing I've noticed is that you unsealed ResolveRequestContext in those changes; can you try putting that back? Sealed classes can sometimes perform better than unsealed ones.

@alsami
Copy link
Member Author

alsami commented Jul 27, 2020

One thing I've noticed is that you unsealed ResolveRequestContext in those changes; can you try putting that back? Sealed classes can sometimes perform better than unsealed ones.

Will look into it tonight, since there is the class MockPipelineRequestContext, which derived previously from ResolveRequestContextBase. Now that ResolveRequestContextBase is all abstract, that's not an easy change.

@alistairjevans
Copy link
Member

Actually, I've also realised that all of the properties in ResolveRequestContextBase are now virtual (because they're abstract), meaning they will employ a similar vtable lookup to an interface. Before, most of the properties in the ResolveRequestContextBase were not virtual, so they would have been faster.

That virtual property access may be what's killing us.

It may come down to changing the ResolveRequestContextBase back to taking constructor parameters.

@alsami
Copy link
Member Author

alsami commented Jul 27, 2020

It may come down to changing the ResolveRequestContextBase back to taking constructor parameters.

Ah, jeez. Will give it a shot later and run the benchmarks again locally. Let's see if that's the cause. If so, we need something that's more testable.

Basically that would mean something like this, if we were to make it more testable this time.
ResolveRequestContextBase -> DefaultResolveRequestContext
ResolveRequestContextBase -> ResolveRequestContext

@tillig
Copy link
Member

tillig commented Jul 27, 2020

I admit I'm only sorta following all this. With the other reviews and such going on, I haven't had the ability to dive in and think I'd just be a distraction. The ultimate goal of having testable extensions stands.

On that note, I'm holding on the DOT graph thing until this settles out. It's not a rush; I'd rather get it right than get it fast. But I don't see the value in updating it on each of these changes until we're happy with them.

@alsami
Copy link
Member Author

alsami commented Jul 28, 2020

But I don't see the value in updating it on each of these changes until we're happy with them.

You mean you would make all of the changes requested in #1164 with a single PR?

@autofac autofac deleted a comment from autofac-bot bot Jul 29, 2020
@alistairjevans
Copy link
Member

Pretty much, yeah.

@alistairjevans
Copy link
Member

Any 'Testable' classes would go into the test projects though. Or we just use Moq, which has recently been added to the project.

@alsami alsami force-pushed the feat/bring_back_base_resolverequestcontext branch from 57fb091 to c3e40dc Compare August 3, 2020 17:10
@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

@alistairjevans I've applied the changes as we have agreed on.

Very interesting is that the test RepeatedResolveWhileTheScopeIsDisposing_ObjectDisposedExceptionThrownOnly, has failed for the past two pipeline. Here and here. I've executed it locally many times, can't get it to fail.

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

@autofac-bot benchmark DeepGraphResolveBenchmark

@autofac-bot
Copy link

autofac-bot bot commented Aug 3, 2020

DeepGraphResolveBenchmark

Target: v6

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Resolve 17.70 us 0.349 us 0.441 us 2.3804 0.0610 - 19.49 KB
Complete benchmark output
// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// ***** Done, took 00:00:00 (0.05 sec)   *****
// Found 1 benchmarks:
//   DeepGraphResolveBenchmark.Resolve: Job-HONCPY(Toolchain=InProcessEmitToolchain)

// **************************
// Benchmark: DeepGraphResolveBenchmark.Resolve: Job-HONCPY(Toolchain=InProcessEmitToolchain)
// *** Execute ***
// Launch: 1 / 1

// Benchmark Process Environment Information:
// Runtime=.NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT
// GC=Concurrent Workstation
// Job: Job-HONCPY(Toolchain=InProcessEmitToolchain)

OverheadJitting  1: 1 op, 98900.00 ns, 98.9000 us/op
WorkloadJitting  1: 1 op, 8575900.00 ns, 8.5759 ms/op

OverheadJitting  2: 16 op, 153300.00 ns, 9.5813 us/op
WorkloadJitting  2: 16 op, 868200.00 ns, 54.2625 us/op

WorkloadPilot    1: 16 op, 585700.00 ns, 36.6063 us/op
WorkloadPilot    2: 32 op, 1116900.00 ns, 34.9031 us/op
WorkloadPilot    3: 64 op, 2139100.00 ns, 33.4234 us/op
WorkloadPilot    4: 128 op, 4408800.00 ns, 34.4438 us/op
WorkloadPilot    5: 256 op, 9455100.00 ns, 36.9340 us/op
WorkloadPilot    6: 512 op, 17925000.00 ns, 35.0098 us/op
WorkloadPilot    7: 1024 op, 34779800.00 ns, 33.9646 us/op
WorkloadPilot    8: 2048 op, 69160600.00 ns, 33.7698 us/op
WorkloadPilot    9: 4096 op, 143350300.00 ns, 34.9976 us/op
WorkloadPilot   10: 8192 op, 175147100.00 ns, 21.3803 us/op
WorkloadPilot   11: 16384 op, 292524700.00 ns, 17.8543 us/op
WorkloadPilot   12: 32768 op, 577563000.00 ns, 17.6258 us/op

OverheadWarmup   1: 32768 op, 42800.00 ns, 1.3062 ns/op
OverheadWarmup   2: 32768 op, 40600.00 ns, 1.2390 ns/op
OverheadWarmup   3: 32768 op, 40600.00 ns, 1.2390 ns/op
OverheadWarmup   4: 32768 op, 40700.00 ns, 1.2421 ns/op
OverheadWarmup   5: 32768 op, 40700.00 ns, 1.2421 ns/op
OverheadWarmup   6: 32768 op, 40500.00 ns, 1.2360 ns/op

OverheadActual   1: 32768 op, 40700.00 ns, 1.2421 ns/op
OverheadActual   2: 32768 op, 41200.00 ns, 1.2573 ns/op
OverheadActual   3: 32768 op, 40400.00 ns, 1.2329 ns/op
OverheadActual   4: 32768 op, 40700.00 ns, 1.2421 ns/op
OverheadActual   5: 32768 op, 40600.00 ns, 1.2390 ns/op
OverheadActual   6: 32768 op, 40600.00 ns, 1.2390 ns/op
OverheadActual   7: 32768 op, 40500.00 ns, 1.2360 ns/op
OverheadActual   8: 32768 op, 40800.00 ns, 1.2451 ns/op
OverheadActual   9: 32768 op, 40500.00 ns, 1.2360 ns/op
OverheadActual  10: 32768 op, 40500.00 ns, 1.2360 ns/op
OverheadActual  11: 32768 op, 40600.00 ns, 1.2390 ns/op
OverheadActual  12: 32768 op, 40500.00 ns, 1.2360 ns/op
OverheadActual  13: 32768 op, 40500.00 ns, 1.2360 ns/op
OverheadActual  14: 32768 op, 40500.00 ns, 1.2360 ns/op
OverheadActual  15: 32768 op, 40500.00 ns, 1.2360 ns/op

WorkloadWarmup   1: 32768 op, 562794700.00 ns, 17.1751 us/op
WorkloadWarmup   2: 32768 op, 582930100.00 ns, 17.7896 us/op
WorkloadWarmup   3: 32768 op, 599000000.00 ns, 18.2800 us/op
WorkloadWarmup   4: 32768 op, 583913100.00 ns, 17.8196 us/op
WorkloadWarmup   5: 32768 op, 593411100.00 ns, 18.1095 us/op
WorkloadWarmup   6: 32768 op, 589979500.00 ns, 18.0047 us/op

WorkloadActual   1: 32768 op, 586116600.00 ns, 17.8869 us/op
WorkloadActual   2: 32768 op, 588796300.00 ns, 17.9686 us/op
WorkloadActual   3: 32768 op, 586913800.00 ns, 17.9112 us/op
WorkloadActual   4: 32768 op, 584021700.00 ns, 17.8229 us/op
WorkloadActual   5: 32768 op, 587553000.00 ns, 17.9307 us/op
WorkloadActual   6: 32768 op, 590470000.00 ns, 18.0197 us/op
WorkloadActual   7: 32768 op, 594799400.00 ns, 18.1518 us/op
WorkloadActual   8: 32768 op, 592872000.00 ns, 18.0930 us/op
WorkloadActual   9: 32768 op, 550535900.00 ns, 16.8010 us/op
WorkloadActual  10: 32768 op, 557030500.00 ns, 16.9992 us/op
WorkloadActual  11: 32768 op, 555444900.00 ns, 16.9508 us/op
WorkloadActual  12: 32768 op, 558396900.00 ns, 17.0409 us/op
WorkloadActual  13: 32768 op, 555989200.00 ns, 16.9674 us/op
WorkloadActual  14: 32768 op, 579198000.00 ns, 17.6757 us/op
WorkloadActual  15: 32768 op, 587323500.00 ns, 17.9237 us/op
WorkloadActual  16: 32768 op, 591216400.00 ns, 18.0425 us/op
WorkloadActual  17: 32768 op, 599732900.00 ns, 18.3024 us/op
WorkloadActual  18: 32768 op, 582610000.00 ns, 17.7798 us/op
WorkloadActual  19: 32768 op, 575563300.00 ns, 17.5648 us/op
WorkloadActual  20: 32768 op, 580125900.00 ns, 17.7040 us/op
WorkloadActual  21: 32768 op, 581013900.00 ns, 17.7311 us/op
WorkloadActual  22: 32768 op, 577354100.00 ns, 17.6194 us/op
WorkloadActual  23: 32768 op, 594958600.00 ns, 18.1567 us/op

WorkloadResult   1: 32768 op, 586076100.00 ns, 17.8856 us/op
WorkloadResult   2: 32768 op, 588755800.00 ns, 17.9674 us/op
WorkloadResult   3: 32768 op, 586873300.00 ns, 17.9100 us/op
WorkloadResult   4: 32768 op, 583981200.00 ns, 17.8217 us/op
WorkloadResult   5: 32768 op, 587512500.00 ns, 17.9295 us/op
WorkloadResult   6: 32768 op, 590429500.00 ns, 18.0185 us/op
WorkloadResult   7: 32768 op, 594758900.00 ns, 18.1506 us/op
WorkloadResult   8: 32768 op, 592831500.00 ns, 18.0918 us/op
WorkloadResult   9: 32768 op, 550495400.00 ns, 16.7998 us/op
WorkloadResult  10: 32768 op, 556990000.00 ns, 16.9980 us/op
WorkloadResult  11: 32768 op, 555404400.00 ns, 16.9496 us/op
WorkloadResult  12: 32768 op, 558356400.00 ns, 17.0397 us/op
WorkloadResult  13: 32768 op, 555948700.00 ns, 16.9662 us/op
WorkloadResult  14: 32768 op, 579157500.00 ns, 17.6745 us/op
WorkloadResult  15: 32768 op, 587283000.00 ns, 17.9225 us/op
WorkloadResult  16: 32768 op, 591175900.00 ns, 18.0413 us/op
WorkloadResult  17: 32768 op, 599692400.00 ns, 18.3012 us/op
WorkloadResult  18: 32768 op, 582569500.00 ns, 17.7786 us/op
WorkloadResult  19: 32768 op, 575522800.00 ns, 17.5636 us/op
WorkloadResult  20: 32768 op, 580085400.00 ns, 17.7028 us/op
WorkloadResult  21: 32768 op, 580973400.00 ns, 17.7299 us/op
WorkloadResult  22: 32768 op, 577313600.00 ns, 17.6182 us/op
WorkloadResult  23: 32768 op, 594918100.00 ns, 18.1555 us/op
GC:  78 2 0 654049280 32768
Threading:  2 0 32768


Mean = 17.696 us, StdErr = 0.092 us (0.52%), N = 23, StdDev = 0.441 us
Min = 16.800 us, Q1 = 17.591 us, Median = 17.822 us, Q3 = 17.993 us, Max = 18.301 us
IQR = 0.402 us, LowerFence = 16.988 us, UpperFence = 18.596 us
ConfidenceInterval = [17.347 us; 18.045 us] (CI 99.9%), Margin = 0.349 us (1.97% of Mean)
Skewness = -0.78, Kurtosis = 2.24, MValue = 2

// ***** BenchmarkRunner: Finish  *****

// * Export *
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-39\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report.csv
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-39\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report-github.md
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-39\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report.html

// * Detailed results *
DeepGraphResolveBenchmark.Resolve: Job-HONCPY(Toolchain=InProcessEmitToolchain)
Runtime = ; GC = 
Mean = 17.696 us, StdErr = 0.092 us (0.52%), N = 23, StdDev = 0.441 us
Min = 16.800 us, Q1 = 17.591 us, Median = 17.822 us, Q3 = 17.993 us, Max = 18.301 us
IQR = 0.402 us, LowerFence = 16.988 us, UpperFence = 18.596 us
ConfidenceInterval = [17.347 us; 18.045 us] (CI 99.9%), Margin = 0.349 us (1.97% of Mean)
Skewness = -0.78, Kurtosis = 2.24, MValue = 2
-------------------- Histogram --------------------
[16.716 us ; 17.123 us) | @@@@@
[17.123 us ; 17.654 us) | @@
[17.654 us ; 18.062 us) | @@@@@@@@@@@@
[18.062 us ; 18.505 us) | @@@@
---------------------------------------------------

// * Summary *

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.388 (2004/?/20H1)
AMD Ryzen 9 3950X, 1 CPU, 32 logical and 16 physical cores
.NET Core SDK=3.1.302
  [Host] : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT

Toolchain=InProcessEmitToolchain  

|  Method |     Mean |    Error |   StdDev |  Gen 0 |  Gen 1 | Gen 2 | Allocated |
|-------- |---------:|---------:|---------:|-------:|-------:|------:|----------:|
| Resolve | 17.70 us | 0.349 us | 0.441 us | 2.3804 | 0.0610 |     - |  19.49 KB |

// * Legends *
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Gen 0     : GC Generation 0 collects per 1000 operations
  Gen 1     : GC Generation 1 collects per 1000 operations
  Gen 2     : GC Generation 2 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 us      : 1 Microsecond (0.000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
// ** Remained 0 benchmark(s) to run **
Run time: 00:00:19 (19.28 sec), executed benchmarks: 1

Global total time: 00:00:19 (19.33 sec), executed benchmarks: 1
// * Artifacts cleanup *

Source: feat/bring_back_base_resolverequestcontext

Method Mean Error StdDev Median Gen 0 Gen 1 Gen 2 Allocated
Resolve 15.14 us 0.301 us 0.503 us 14.84 us 2.0599 0.0458 - 16.91 KB
Complete benchmark output
// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// ***** Done, took 00:00:00 (0.04 sec)   *****
// Found 1 benchmarks:
//   DeepGraphResolveBenchmark.Resolve: Job-NZBDEZ(Toolchain=InProcessEmitToolchain)

// **************************
// Benchmark: DeepGraphResolveBenchmark.Resolve: Job-NZBDEZ(Toolchain=InProcessEmitToolchain)
// *** Execute ***
// Launch: 1 / 1

// Benchmark Process Environment Information:
// Runtime=.NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT
// GC=Concurrent Workstation
// Job: Job-NZBDEZ(Toolchain=InProcessEmitToolchain)

OverheadJitting  1: 1 op, 93700.00 ns, 93.7000 us/op
WorkloadJitting  1: 1 op, 8827200.00 ns, 8.8272 ms/op

OverheadJitting  2: 16 op, 174000.00 ns, 10.8750 us/op
WorkloadJitting  2: 16 op, 777100.00 ns, 48.5688 us/op

WorkloadPilot    1: 16 op, 484100.00 ns, 30.2563 us/op
WorkloadPilot    2: 32 op, 984000.00 ns, 30.7500 us/op
WorkloadPilot    3: 64 op, 1972700.00 ns, 30.8234 us/op
WorkloadPilot    4: 128 op, 3832300.00 ns, 29.9398 us/op
WorkloadPilot    5: 256 op, 8082800.00 ns, 31.5734 us/op
WorkloadPilot    6: 512 op, 16184800.00 ns, 31.6109 us/op
WorkloadPilot    7: 1024 op, 29702700.00 ns, 29.0065 us/op
WorkloadPilot    8: 2048 op, 59132100.00 ns, 28.8731 us/op
WorkloadPilot    9: 4096 op, 61375900.00 ns, 14.9844 us/op
WorkloadPilot   10: 8192 op, 121081200.00 ns, 14.7804 us/op
WorkloadPilot   11: 16384 op, 244253900.00 ns, 14.9081 us/op
WorkloadPilot   12: 32768 op, 484024600.00 ns, 14.7713 us/op
WorkloadPilot   13: 65536 op, 967780700.00 ns, 14.7672 us/op

OverheadWarmup   1: 65536 op, 82600.00 ns, 1.2604 ns/op
OverheadWarmup   2: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadWarmup   3: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadWarmup   4: 65536 op, 101300.00 ns, 1.5457 ns/op
OverheadWarmup   5: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadWarmup   6: 65536 op, 80400.00 ns, 1.2268 ns/op

OverheadActual   1: 65536 op, 81100.00 ns, 1.2375 ns/op
OverheadActual   2: 65536 op, 80500.00 ns, 1.2283 ns/op
OverheadActual   3: 65536 op, 80600.00 ns, 1.2299 ns/op
OverheadActual   4: 65536 op, 81900.00 ns, 1.2497 ns/op
OverheadActual   5: 65536 op, 80600.00 ns, 1.2299 ns/op
OverheadActual   6: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadActual   7: 65536 op, 80500.00 ns, 1.2283 ns/op
OverheadActual   8: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadActual   9: 65536 op, 81200.00 ns, 1.2390 ns/op
OverheadActual  10: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadActual  11: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadActual  12: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadActual  13: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual  14: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual  15: 65536 op, 80300.00 ns, 1.2253 ns/op

WorkloadWarmup   1: 65536 op, 969061600.00 ns, 14.7867 us/op
WorkloadWarmup   2: 65536 op, 963476700.00 ns, 14.7015 us/op
WorkloadWarmup   3: 65536 op, 968906900.00 ns, 14.7843 us/op
WorkloadWarmup   4: 65536 op, 966145000.00 ns, 14.7422 us/op
WorkloadWarmup   5: 65536 op, 970888800.00 ns, 14.8146 us/op
WorkloadWarmup   6: 65536 op, 964433800.00 ns, 14.7161 us/op

WorkloadActual   1: 65536 op, 1011431300.00 ns, 15.4332 us/op
WorkloadActual   2: 65536 op, 1038763100.00 ns, 15.8503 us/op
WorkloadActual   3: 65536 op, 1049658500.00 ns, 16.0165 us/op
WorkloadActual   4: 65536 op, 1047506100.00 ns, 15.9837 us/op
WorkloadActual   5: 65536 op, 1049935900.00 ns, 16.0208 us/op
WorkloadActual   6: 65536 op, 1047464100.00 ns, 15.9830 us/op
WorkloadActual   7: 65536 op, 1058327800.00 ns, 16.1488 us/op
WorkloadActual   8: 65536 op, 1032262700.00 ns, 15.7511 us/op
WorkloadActual   9: 65536 op, 1029498800.00 ns, 15.7089 us/op
WorkloadActual  10: 65536 op, 1038738600.00 ns, 15.8499 us/op
WorkloadActual  11: 65536 op, 1032093800.00 ns, 15.7485 us/op
WorkloadActual  12: 65536 op, 969334100.00 ns, 14.7909 us/op
WorkloadActual  13: 65536 op, 961592500.00 ns, 14.6727 us/op
WorkloadActual  14: 65536 op, 971063400.00 ns, 14.8173 us/op
WorkloadActual  15: 65536 op, 977833000.00 ns, 14.9205 us/op
WorkloadActual  16: 65536 op, 965169600.00 ns, 14.7273 us/op
WorkloadActual  17: 65536 op, 972669400.00 ns, 14.8418 us/op
WorkloadActual  18: 65536 op, 966936500.00 ns, 14.7543 us/op
WorkloadActual  19: 65536 op, 969854700.00 ns, 14.7988 us/op
WorkloadActual  20: 65536 op, 965031700.00 ns, 14.7252 us/op
WorkloadActual  21: 65536 op, 961769300.00 ns, 14.6754 us/op
WorkloadActual  22: 65536 op, 970146200.00 ns, 14.8033 us/op
WorkloadActual  23: 65536 op, 972578700.00 ns, 14.8404 us/op
WorkloadActual  24: 65536 op, 974386900.00 ns, 14.8680 us/op
WorkloadActual  25: 65536 op, 971999400.00 ns, 14.8315 us/op
WorkloadActual  26: 65536 op, 970672300.00 ns, 14.8113 us/op
WorkloadActual  27: 65536 op, 971530300.00 ns, 14.8244 us/op
WorkloadActual  28: 65536 op, 981093000.00 ns, 14.9703 us/op
WorkloadActual  29: 65536 op, 972626600.00 ns, 14.8411 us/op
WorkloadActual  30: 65536 op, 972454300.00 ns, 14.8385 us/op
WorkloadActual  31: 65536 op, 969028000.00 ns, 14.7862 us/op
WorkloadActual  32: 65536 op, 979885200.00 ns, 14.9519 us/op
WorkloadActual  33: 65536 op, 973969200.00 ns, 14.8616 us/op
WorkloadActual  34: 65536 op, 968168800.00 ns, 14.7731 us/op
WorkloadActual  35: 65536 op, 971035500.00 ns, 14.8168 us/op
WorkloadActual  36: 65536 op, 976943800.00 ns, 14.9070 us/op

WorkloadResult   1: 65536 op, 1011350900.00 ns, 15.4320 us/op
WorkloadResult   2: 65536 op, 1038682700.00 ns, 15.8490 us/op
WorkloadResult   3: 65536 op, 1049578100.00 ns, 16.0153 us/op
WorkloadResult   4: 65536 op, 1047425700.00 ns, 15.9824 us/op
WorkloadResult   5: 65536 op, 1049855500.00 ns, 16.0195 us/op
WorkloadResult   6: 65536 op, 1047383700.00 ns, 15.9818 us/op
WorkloadResult   7: 65536 op, 1058247400.00 ns, 16.1476 us/op
WorkloadResult   8: 65536 op, 1032182300.00 ns, 15.7499 us/op
WorkloadResult   9: 65536 op, 1029418400.00 ns, 15.7077 us/op
WorkloadResult  10: 65536 op, 1038658200.00 ns, 15.8487 us/op
WorkloadResult  11: 65536 op, 1032013400.00 ns, 15.7473 us/op
WorkloadResult  12: 65536 op, 969253700.00 ns, 14.7896 us/op
WorkloadResult  13: 65536 op, 961512100.00 ns, 14.6715 us/op
WorkloadResult  14: 65536 op, 970983000.00 ns, 14.8160 us/op
WorkloadResult  15: 65536 op, 977752600.00 ns, 14.9193 us/op
WorkloadResult  16: 65536 op, 965089200.00 ns, 14.7261 us/op
WorkloadResult  17: 65536 op, 972589000.00 ns, 14.8405 us/op
WorkloadResult  18: 65536 op, 966856100.00 ns, 14.7531 us/op
WorkloadResult  19: 65536 op, 969774300.00 ns, 14.7976 us/op
WorkloadResult  20: 65536 op, 964951300.00 ns, 14.7240 us/op
WorkloadResult  21: 65536 op, 961688900.00 ns, 14.6742 us/op
WorkloadResult  22: 65536 op, 970065800.00 ns, 14.8020 us/op
WorkloadResult  23: 65536 op, 972498300.00 ns, 14.8391 us/op
WorkloadResult  24: 65536 op, 974306500.00 ns, 14.8667 us/op
WorkloadResult  25: 65536 op, 971919000.00 ns, 14.8303 us/op
WorkloadResult  26: 65536 op, 970591900.00 ns, 14.8101 us/op
WorkloadResult  27: 65536 op, 971449900.00 ns, 14.8231 us/op
WorkloadResult  28: 65536 op, 981012600.00 ns, 14.9691 us/op
WorkloadResult  29: 65536 op, 972546200.00 ns, 14.8399 us/op
WorkloadResult  30: 65536 op, 972373900.00 ns, 14.8372 us/op
WorkloadResult  31: 65536 op, 968947600.00 ns, 14.7850 us/op
WorkloadResult  32: 65536 op, 979804800.00 ns, 14.9506 us/op
WorkloadResult  33: 65536 op, 973888800.00 ns, 14.8604 us/op
WorkloadResult  34: 65536 op, 968088400.00 ns, 14.7719 us/op
WorkloadResult  35: 65536 op, 970955100.00 ns, 14.8156 us/op
WorkloadResult  36: 65536 op, 976863400.00 ns, 14.9058 us/op
GC:  135 3 0 1135083520 65536
Threading:  2 0 65536


Mean = 15.136 us, StdErr = 0.084 us (0.55%), N = 36, StdDev = 0.503 us
Min = 14.672 us, Q1 = 14.801 us, Median = 14.840 us, Q3 = 15.718 us, Max = 16.148 us
IQR = 0.917 us, LowerFence = 13.426 us, UpperFence = 17.093 us
ConfidenceInterval = [14.835 us; 15.437 us] (CI 99.9%), Margin = 0.301 us (1.99% of Mean)
Skewness = 0.89, Kurtosis = 1.96, MValue = 2.72

// ***** BenchmarkRunner: Finish  *****

// * Export *
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-13\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report.csv
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-13\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report-github.md
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-13\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report.html

// * Detailed results *
DeepGraphResolveBenchmark.Resolve: Job-NZBDEZ(Toolchain=InProcessEmitToolchain)
Runtime = ; GC = 
Mean = 15.136 us, StdErr = 0.084 us (0.55%), N = 36, StdDev = 0.503 us
Min = 14.672 us, Q1 = 14.801 us, Median = 14.840 us, Q3 = 15.718 us, Max = 16.148 us
IQR = 0.917 us, LowerFence = 13.426 us, UpperFence = 17.093 us
ConfidenceInterval = [14.835 us; 15.437 us] (CI 99.9%), Margin = 0.301 us (1.99% of Mean)
Skewness = 0.89, Kurtosis = 1.96, MValue = 2.72
-------------------- Histogram --------------------
[14.620 us ; 15.232 us) | @@@@@@@@@@@@@@@@@@@@@@@@@
[15.232 us ; 15.663 us) | @
[15.663 us ; 16.348 us) | @@@@@@@@@@
---------------------------------------------------

// * Summary *

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.388 (2004/?/20H1)
AMD Ryzen 9 3950X, 1 CPU, 32 logical and 16 physical cores
.NET Core SDK=3.1.302
  [Host] : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT

Toolchain=InProcessEmitToolchain  

|  Method |     Mean |    Error |   StdDev |   Median |  Gen 0 |  Gen 1 | Gen 2 | Allocated |
|-------- |---------:|---------:|---------:|---------:|-------:|-------:|------:|----------:|
| Resolve | 15.14 us | 0.301 us | 0.503 us | 14.84 us | 2.0599 | 0.0458 |     - |  16.91 KB |

// * Legends *
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Median    : Value separating the higher half of all measurements (50th percentile)
  Gen 0     : GC Generation 0 collects per 1000 operations
  Gen 1     : GC Generation 1 collects per 1000 operations
  Gen 2     : GC Generation 2 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 us      : 1 Microsecond (0.000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
// ** Remained 0 benchmark(s) to run **
Run time: 00:00:45 (45.07 sec), executed benchmarks: 1

Global total time: 00:00:45 (45.11 sec), executed benchmarks: 1
// * Artifacts cleanup *

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

Benchmarks looking good.

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

@autofac-bot benchmark DeepGraphResolveBenchmark commit:944d4cb4b6b630f70229daf9de7726d5a3dbe848

@autofac-bot
Copy link

autofac-bot bot commented Aug 3, 2020

DeepGraphResolveBenchmark

Target: 944d4cb

Unhandled exception. System.ArgumentOutOfRangeException: Length cannot be less than zero. (Parameter 'length')
at System.String.Substring(Int32 startIndex, Int32 length)
at Autofac.Benchmarks.BenchmarkConfig..ctor() in C:\Users\sami\AppData\Local.autofacbot\ea8f00cda1286d46add852adb648dd8d\Target\src\bench\Autofac.Benchmarks\BenchmarkConfig.cs:line 13
at Autofac.Benchmarks.Program.Main(String[] args) in C:\Users\sami\AppData\Local.autofacbot\ea8f00cda1286d46add852adb648dd8d\Target\src\bench\Autofac.Benchmarks\Program.cs:line 8

Complete benchmark output
Unhandled exception. System.ArgumentOutOfRangeException: Length cannot be less than zero. (Parameter 'length')
   at System.String.Substring(Int32 startIndex, Int32 length)
   at Autofac.Benchmarks.BenchmarkConfig..ctor() in C:\Users\sami\AppData\Local\.autofacbot\ea8f00cda1286d46add852adb648dd8d\Target\src\bench\Autofac.Benchmarks\BenchmarkConfig.cs:line 13
   at Autofac.Benchmarks.Program.Main(String[] args) in C:\Users\sami\AppData\Local\.autofacbot\ea8f00cda1286d46add852adb648dd8d\Target\src\bench\Autofac.Benchmarks\Program.cs:line 8

Source: feat/bring_back_base_resolverequestcontext

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
Resolve 14.67 us 0.121 us 0.113 us 2.0599 0.0458 - 16.91 KB
Complete benchmark output
// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
// ***** Done, took 00:00:00 (0.04 sec)   *****
// Found 1 benchmarks:
//   DeepGraphResolveBenchmark.Resolve: Job-XODYYH(Toolchain=InProcessEmitToolchain)

// **************************
// Benchmark: DeepGraphResolveBenchmark.Resolve: Job-XODYYH(Toolchain=InProcessEmitToolchain)
// *** Execute ***
// Launch: 1 / 1

// Benchmark Process Environment Information:
// Runtime=.NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT
// GC=Concurrent Workstation
// Job: Job-XODYYH(Toolchain=InProcessEmitToolchain)

OverheadJitting  1: 1 op, 93800.00 ns, 93.8000 us/op
WorkloadJitting  1: 1 op, 8659700.00 ns, 8.6597 ms/op

OverheadJitting  2: 16 op, 228600.00 ns, 14.2875 us/op
WorkloadJitting  2: 16 op, 684800.00 ns, 42.8000 us/op

WorkloadPilot    1: 16 op, 470000.00 ns, 29.3750 us/op
WorkloadPilot    2: 32 op, 922900.00 ns, 28.8406 us/op
WorkloadPilot    3: 64 op, 1891100.00 ns, 29.5484 us/op
WorkloadPilot    4: 128 op, 3828900.00 ns, 29.9133 us/op
WorkloadPilot    5: 256 op, 8043500.00 ns, 31.4199 us/op
WorkloadPilot    6: 512 op, 15989200.00 ns, 31.2289 us/op
WorkloadPilot    7: 1024 op, 29908100.00 ns, 29.2071 us/op
WorkloadPilot    8: 2048 op, 57970000.00 ns, 28.3057 us/op
WorkloadPilot    9: 4096 op, 107636800.00 ns, 26.2785 us/op
WorkloadPilot   10: 8192 op, 120502100.00 ns, 14.7097 us/op
WorkloadPilot   11: 16384 op, 240041200.00 ns, 14.6510 us/op
WorkloadPilot   12: 32768 op, 469933500.00 ns, 14.3412 us/op
WorkloadPilot   13: 65536 op, 944747000.00 ns, 14.4157 us/op

OverheadWarmup   1: 65536 op, 81900.00 ns, 1.2497 ns/op
OverheadWarmup   2: 65536 op, 80500.00 ns, 1.2283 ns/op
OverheadWarmup   3: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadWarmup   4: 65536 op, 96900.00 ns, 1.4786 ns/op
OverheadWarmup   5: 65536 op, 80500.00 ns, 1.2283 ns/op
OverheadWarmup   6: 65536 op, 79900.00 ns, 1.2192 ns/op
OverheadWarmup   7: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadWarmup   8: 65536 op, 80300.00 ns, 1.2253 ns/op

OverheadActual   1: 65536 op, 80100.00 ns, 1.2222 ns/op
OverheadActual   2: 65536 op, 80000.00 ns, 1.2207 ns/op
OverheadActual   3: 65536 op, 80100.00 ns, 1.2222 ns/op
OverheadActual   4: 65536 op, 79600.00 ns, 1.2146 ns/op
OverheadActual   5: 65536 op, 80500.00 ns, 1.2283 ns/op
OverheadActual   6: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual   7: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual   8: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual   9: 65536 op, 121900.00 ns, 1.8600 ns/op
OverheadActual  10: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadActual  11: 65536 op, 80400.00 ns, 1.2268 ns/op
OverheadActual  12: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual  13: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual  14: 65536 op, 80300.00 ns, 1.2253 ns/op
OverheadActual  15: 65536 op, 80300.00 ns, 1.2253 ns/op

WorkloadWarmup   1: 65536 op, 970223200.00 ns, 14.8044 us/op
WorkloadWarmup   2: 65536 op, 981402500.00 ns, 14.9750 us/op
WorkloadWarmup   3: 65536 op, 973011600.00 ns, 14.8470 us/op
WorkloadWarmup   4: 65536 op, 965213000.00 ns, 14.7280 us/op
WorkloadWarmup   5: 65536 op, 966632200.00 ns, 14.7496 us/op
WorkloadWarmup   6: 65536 op, 980152100.00 ns, 14.9559 us/op
WorkloadWarmup   7: 65536 op, 973871000.00 ns, 14.8601 us/op

WorkloadActual   1: 65536 op, 960307700.00 ns, 14.6531 us/op
WorkloadActual   2: 65536 op, 958360300.00 ns, 14.6234 us/op
WorkloadActual   3: 65536 op, 948760600.00 ns, 14.4769 us/op
WorkloadActual   4: 65536 op, 954979200.00 ns, 14.5718 us/op
WorkloadActual   5: 65536 op, 954330000.00 ns, 14.5619 us/op
WorkloadActual   6: 65536 op, 953892700.00 ns, 14.5552 us/op
WorkloadActual   7: 65536 op, 967432400.00 ns, 14.7618 us/op
WorkloadActual   8: 65536 op, 955544400.00 ns, 14.5805 us/op
WorkloadActual   9: 65536 op, 965694400.00 ns, 14.7353 us/op
WorkloadActual  10: 65536 op, 972709700.00 ns, 14.8424 us/op
WorkloadActual  11: 65536 op, 973757500.00 ns, 14.8584 us/op
WorkloadActual  12: 65536 op, 970574400.00 ns, 14.8098 us/op
WorkloadActual  13: 65536 op, 960596000.00 ns, 14.6575 us/op
WorkloadActual  14: 65536 op, 960210800.00 ns, 14.6517 us/op
WorkloadActual  15: 65536 op, 960746700.00 ns, 14.6598 us/op

WorkloadResult   1: 65536 op, 960227400.00 ns, 14.6519 us/op
WorkloadResult   2: 65536 op, 958280000.00 ns, 14.6222 us/op
WorkloadResult   3: 65536 op, 948680300.00 ns, 14.4757 us/op
WorkloadResult   4: 65536 op, 954898900.00 ns, 14.5706 us/op
WorkloadResult   5: 65536 op, 954249700.00 ns, 14.5607 us/op
WorkloadResult   6: 65536 op, 953812400.00 ns, 14.5540 us/op
WorkloadResult   7: 65536 op, 967352100.00 ns, 14.7606 us/op
WorkloadResult   8: 65536 op, 955464100.00 ns, 14.5792 us/op
WorkloadResult   9: 65536 op, 965614100.00 ns, 14.7341 us/op
WorkloadResult  10: 65536 op, 972629400.00 ns, 14.8411 us/op
WorkloadResult  11: 65536 op, 973677200.00 ns, 14.8571 us/op
WorkloadResult  12: 65536 op, 970494100.00 ns, 14.8086 us/op
WorkloadResult  13: 65536 op, 960515700.00 ns, 14.6563 us/op
WorkloadResult  14: 65536 op, 960130500.00 ns, 14.6504 us/op
WorkloadResult  15: 65536 op, 960666400.00 ns, 14.6586 us/op
GC:  135 3 0 1135083520 65536
Threading:  2 0 65536


Mean = 14.665 us, StdErr = 0.029 us (0.20%), N = 15, StdDev = 0.113 us
Min = 14.476 us, Q1 = 14.575 us, Median = 14.652 us, Q3 = 14.747 us, Max = 14.857 us
IQR = 0.172 us, LowerFence = 14.316 us, UpperFence = 15.006 us
ConfidenceInterval = [14.544 us; 14.787 us] (CI 99.9%), Margin = 0.121 us (0.83% of Mean)
Skewness = 0.26, Kurtosis = 1.83, MValue = 2

// ***** BenchmarkRunner: Finish  *****

// * Export *
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-46\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report.csv
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-46\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report-github.md
  BenchmarkDotNet.Artifacts\03-08-2020_05-08-46\results\Autofac.Benchmarks.DeepGraphResolveBenchmark-report.html

// * Detailed results *
DeepGraphResolveBenchmark.Resolve: Job-XODYYH(Toolchain=InProcessEmitToolchain)
Runtime = ; GC = 
Mean = 14.665 us, StdErr = 0.029 us (0.20%), N = 15, StdDev = 0.113 us
Min = 14.476 us, Q1 = 14.575 us, Median = 14.652 us, Q3 = 14.747 us, Max = 14.857 us
IQR = 0.172 us, LowerFence = 14.316 us, UpperFence = 15.006 us
ConfidenceInterval = [14.544 us; 14.787 us] (CI 99.9%), Margin = 0.121 us (0.83% of Mean)
Skewness = 0.26, Kurtosis = 1.83, MValue = 2
-------------------- Histogram --------------------
[14.460 us ; 14.917 us) | @@@@@@@@@@@@@@@
---------------------------------------------------

// * Summary *

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.388 (2004/?/20H1)
AMD Ryzen 9 3950X, 1 CPU, 32 logical and 16 physical cores
.NET Core SDK=3.1.302
  [Host] : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT

Toolchain=InProcessEmitToolchain  

|  Method |     Mean |    Error |   StdDev |  Gen 0 |  Gen 1 | Gen 2 | Allocated |
|-------- |---------:|---------:|---------:|-------:|-------:|------:|----------:|
| Resolve | 14.67 us | 0.121 us | 0.113 us | 2.0599 | 0.0458 |     - |  16.91 KB |

// * Legends *
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Gen 0     : GC Generation 0 collects per 1000 operations
  Gen 1     : GC Generation 1 collects per 1000 operations
  Gen 2     : GC Generation 2 collects per 1000 operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 us      : 1 Microsecond (0.000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
// ** Remained 0 benchmark(s) to run **
Run time: 00:00:24 (24.74 sec), executed benchmarks: 1

Global total time: 00:00:24 (24.78 sec), executed benchmarks: 1
// * Artifacts cleanup *

@alsami alsami changed the title feat: remove interface for IResolveRequestContext and re-add ResolveRequestContextBase defining all methods as abstract feat: remove interface for IResolveRequestContext Aug 3, 2020
@alsami alsami force-pushed the feat/bring_back_base_resolverequestcontext branch from c3e40dc to 82b643e Compare August 3, 2020 18:10
@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

Test run number3 magically passed 🤷‍♂️

@alsami alsami requested a review from alistairjevans August 3, 2020 18:18
@alistairjevans
Copy link
Member

Great, I love intermittent errors that only happen in CI. /s

Looks like somethings not right with the benchmarks, look like an exception is getting output?

Is there a way to indicate to the bot that we only want the results rather than the full output?

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

Looks like somethings not right with the benchmarks, look like an exception is getting output?

Yeah, it's because I am publishing the benchmarks before executing them. That only works when changes from #1181 are applied. In that specific commit, that's not the case. Meaning, that for those commits, we will still have to run those benchmarks manually.

Is there a way to indicate to the bot that we only want the results rather than the full output?

In this case you mean that you don't want to show the exception, but only the valid results, if they are present? If so, I could change the code to hide the exception details.

@tillig
Copy link
Member

tillig commented Aug 3, 2020

You can use some fancy HTML 5 tags to have collapsible bits if you want.

<details>
<summary>Collapsible panel!</summary>
This is the detail that is hidden by the panel.
</details>

That renders as:

Collapsible panel! This is the detail that is hidden by the panel.

I don't think it makes the emails shorter, but the rendered view gets tidier.

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

Will do, doing so also for the complete output. Will just hide the error instead!

@alistairjevans
Copy link
Member

90% of the time all we'll care about is the summary table from each benchmark. The full verbose output makes it harder to compare.

Any way to only print the result table of each unless we explicitly ask for it to be more verbose?

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

Any way to only print the result table of each unless we explicitly ask for it to be more verbose?

You want the full output in the collapse only if requested?

@alistairjevans
Copy link
Member

Could do, but I'd also be happy with a /verbose option on the original command.

Basically, I'm not too fussed about the exact format, provided the two result tables are visually adjacent when viewed in GitHub.

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

Sure, can do so, hopefully the upcoming weekend.

For errors it will look like that for now:
#1179 (comment)

@alistairjevans
Copy link
Member

Cool, thanks. 😀

I'll take a look at the updates to this PR tomorrow and get it merged in.

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

Could do, but I'd also be happy with a /verbose option on the original command.

Basically, I'm not too fussed about the exact format, provided the two result tables are visually adjacent when viewed in GitHub.

@tillig would be great if you could clone and add the repositories to the org, so that we can collect issues/feature requests there

@tillig
Copy link
Member

tillig commented Aug 3, 2020

Yup, I'm a little under the gun at work so this isn't going to happen today.

@alsami
Copy link
Member Author

alsami commented Aug 3, 2020

No worries, tyt.

Copy link
Member

@alistairjevans alistairjevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels pretty good now; I'm going to merge it. Thanks @alsami for sticking with this one through all the changes!

@alistairjevans alistairjevans merged commit 58e84c8 into autofac:v6 Aug 4, 2020
@alistairjevans alistairjevans added this to the v6.0 milestone Sep 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants