Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 79 Improvements on 12/10/2022 1:39:11 AM #10695

Closed
performanceautofiler bot opened this issue Dec 13, 2022 · 4 comments
Closed

[Perf] Linux/arm64: 79 Improvements on 12/10/2022 1:39:11 AM #10695

performanceautofiler bot opened this issue Dec 13, 2022 · 4 comments
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Dec 13, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in Interop.StructureToPtr

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MarshalPtrToStructure - Duration of single invocation 7.71 μs 3.41 μs 0.44 0.11 False
MarshalDestroyStructure - Duration of single invocation 2.13 μs 1.67 μs 0.78 0.14 False
MarshalStructureToPtr - Duration of single invocation 1.78 μs 1.30 μs 0.73 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Interop.StructureToPtr*'

Payloads

Baseline
Compare

Histogram

Interop.StructureToPtr.MarshalPtrToStructure

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 85.97173636755564 (T) = (0 -3451.9263683172476) / Math.Sqrt((37164.389813066795 / (33)) + (2969.99471899917 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.5599875996032989 = (7845.06610542135 - 3451.9263683172476) / 7845.06610542135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Interop.StructureToPtr.MarshalDestroyStructure

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 36.71207976672952 (T) = (0 -1676.5345694021146) / Math.Sqrt((2546.7501765275438 / (33)) + (198.6824098640696 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.22536439185757218 = (2164.2880236585506 - 1676.5345694021146) / 2164.2880236585506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Interop.StructureToPtr.MarshalStructureToPtr

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 54.505001981608984 (T) = (0 -1292.4393515116967) / Math.Sqrt((3000.024556588361 / (33)) + (13.884601699394686 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2943675276217562 = (1831.604131192681 - 1292.4393515116967) / 1831.604131192681 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 1.13 ms 781.71 μs 0.69 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.73240666215288 (T) = (0 -788965.3900972771) / Math.Sqrt((223902805.90715772 / (33)) + (105145442.08474953 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3087169027199321 = (1141305.7735702656 - 788965.3900972771) / 1141305.7735702656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCustomAttributesMethodOverrideHitInherit - Duration of single invocation 23.77 μs 17.77 μs 0.75 0.30 False
GetCustomAttributesClassMissInherit - Duration of single invocation 26.83 μs 15.19 μs 0.57 0.22 False
GetCustomAttributesClassHit - Duration of single invocation 17.18 μs 12.97 μs 0.76 0.16 False
GetCustomAttributesMethodBaseMissInherit - Duration of single invocation 17.37 μs 10.73 μs 0.62 0.15 False
GetCustomAttributesMethodBaseMiss - Duration of single invocation 17.16 μs 10.36 μs 0.60 0.14 False
GetCustomAttributesClassMiss - Duration of single invocation 17.73 μs 9.08 μs 0.51 0.14 False
GetCustomAttributesMethodBaseHitInherit - Duration of single invocation 18.90 μs 12.61 μs 0.67 0.15 False
GetCustomAttributesMethodOverrideMissInherit - Duration of single invocation 24.64 μs 17.48 μs 0.71 0.08 False
IsDefinedClassMiss - Duration of single invocation 1.93 μs 1.41 μs 0.73 0.14 False
IsDefinedClassMissInherit - Duration of single invocation 11.58 μs 7.06 μs 0.61 0.03 False
GetCustomAttributesClassHitInherit - Duration of single invocation 42.43 μs 28.16 μs 0.66 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesMethodOverrideHitInherit

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 39.69701050768853 (T) = (0 -17755.869787852727) / Math.Sqrt((848781.8807489843 / (33)) + (289.4417088695564 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.26446968239435 = (24140.228299022237 - 17755.869787852727) / 24140.228299022237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 64.85620873878447 (T) = (0 -15091.766495896893) / Math.Sqrt((692885.529128834 / (32)) + (19534.397049341642 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.43238326290746526 = (26587.951886691077 - 15091.766495896893) / 26587.951886691077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 24.503234221054537 (T) = (0 -13037.644948093772) / Math.Sqrt((861285.8887206583 / (33)) + (8639.458935720735 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.24687021973673356 = (17311.28590285766 - 13037.644948093772) / 17311.28590285766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 36.98598222519713 (T) = (0 -10593.114833127129) / Math.Sqrt((488767.5244169311 / (29)) + (36757.85205936265 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.3959063276904651 = (17535.550062340768 - 10593.114833127129) / 17535.550062340768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 63.594688054111025 (T) = (0 -10331.468502533287) / Math.Sqrt((345382.53394158074 / (33)) + (1995.2986107788465 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.39724787563268427 = (17140.49289063528 - 10331.468502533287) / 17140.49289063528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 41.80388648681255 (T) = (0 -9217.562315412739) / Math.Sqrt((427143.68358193524 / (33)) + (35456.98703517261 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.4426732183477976 = (16538.882786302063 - 9217.562315412739) / 16538.882786302063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.01472381066601 (T) = (0 -12583.354248609603) / Math.Sqrt((767831.7790209649 / (30)) + (1066.2373425324467 / (2))) is greater than 2.0422724562973102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (2) - 2, .975) and 0.3395028222979267 = (19051.336891988212 - 12583.354248609603) / 19051.336891988212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 26.21731687968956 (T) = (0 -17686.350843239445) / Math.Sqrt((602109.1685001137 / (33)) + (83302.4376174045 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2662099052825399 = (24102.738604081907 - 17686.350843239445) / 24102.738604081907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedClassMiss

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 32.2884914410365 (T) = (0 -1397.892707341739) / Math.Sqrt((1989.9629137281722 / (33)) + (394.5938567574875 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2704563326024664 = (1916.1193082908594 - 1397.892707341739) / 1916.1193082908594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedClassMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 68.04821463196667 (T) = (0 -7106.05182729861) / Math.Sqrt((55590.64460399214 / (27)) + (5080.77495705225 / (2))) is greater than 2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (2) - 2, .975) and 0.3937309986495147 = (11720.955238466147 - 7106.05182729861) / 11720.955238466147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.GetCustomAttributesClassHitInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.90500945384042 (T) = (0 -28118.150539718947) / Math.Sqrt((3934469.1873802515 / (33)) + (3982.0855815741843 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.33620602988633735 = (42359.75589670425 - 28118.150539718947) / 42359.75589670425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Transient - Duration of single invocation 78.32 μs 65.35 μs 0.83 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 19.428687874112658 (T) = (0 -65544.03542717994) / Math.Sqrt((15503006.475434586 / (33)) + (78111.06180159663 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.17454116675947892 = (79403.15469140036 - 65544.03542717994) / 79403.15469140036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Field_Set_int - Duration of single invocation 4.15 μs 2.37 μs 0.57 0.22 False
Field_Get_class - Duration of single invocation 3.98 μs 2.15 μs 0.54 0.24 False
Property_Set_class - Duration of single invocation 3.65 μs 2.33 μs 0.64 0.05 False
Property_Set_int - Duration of single invocation 3.25 μs 2.33 μs 0.72 0.22 False
Field_Set_class - Duration of single invocation 5.26 μs 3.36 μs 0.64 0.12 False
Ctor4_int_string_struct_class - Duration of single invocation 5.19 μs 1.42 μs 0.27 0.16 False
StaticMethod4_arrayNotCached_int_string_struct_class - Duration of single invocation 5.29 μs 1.75 μs 0.33 0.11 False
StaticMethod5_arrayNotCached_int_string_struct_class_bool - Duration of single invocation 6.56 μs 2.15 μs 0.33 0.15 False
Ctor4_ActivatorCreateInstance - Duration of single invocation 17.30 μs 9.87 μs 0.57 0.05 False
StaticMethod4_int_string_struct_class - Duration of single invocation 4.87 μs 1.48 μs 0.30 0.14 False
Ctor0_ActivatorCreateInstance_NoParams - Duration of single invocation 4.67 μs 2.03 μs 0.43 0.06 False
Field_Get_int - Duration of single invocation 3.82 μs 2.22 μs 0.58 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Field_Set_int

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 94.28171210988425 (T) = (0 -2365.1292285714285) / Math.Sqrt((9574.102103001274 / (33)) + (0.022911249535226632 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.4044092945137225 = (3971.0647039738287 - 2365.1292285714285) / 3971.0647039738287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Field_Get_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 30.858252467049788 (T) = (0 -2198.109186598558) / Math.Sqrt((8480.062762604577 / (29)) + (4745.959753187335 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.42021407335293137 = (3791.242742490034 - 2198.109186598558) / 3791.242742490034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Property_Set_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 67.20298761488192 (T) = (0 -2341.841537202381) / Math.Sqrt((5457.773203271072 / (33)) + (562.8390143537583 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3775612823183801 = (3762.364825126838 - 2341.841537202381) / 3762.364825126838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Property_Set_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 33.85698777942084 (T) = (0 -2354.7309137436223) / Math.Sqrt((9131.103228658018 / (33)) + (985.6035615584985 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.28512813742694004 = (3293.920263231746 - 2354.7309137436223) / 3293.920263231746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Field_Set_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 28.346151495333345 (T) = (0 -3335.879161089743) / Math.Sqrt((98161.51605461809 / (29)) + (1202.161159120456 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.34916001953091136 = (5125.498219524606 - 3335.879161089743) / 5125.498219524606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Ctor4_int_string_struct_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 120.76967992381098 (T) = (0 -1431.5507488162878) / Math.Sqrt((26057.073132415477 / (33)) + (235.42826704920765 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.7176066483733995 = (5069.350041601477 - 1431.5507488162878) / 5069.350041601477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 96.6695143945589 (T) = (0 -1766.9401969907408) / Math.Sqrt((38930.59785230643 / (33)) + (509.26855641451266 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.6744819005483538 = (5428.0858728508565 - 1766.9401969907408) / 5428.0858728508565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 115.64433967188748 (T) = (0 -2150.030017578125) / Math.Sqrt((45309.65064434444 / (29)) + (0.537326354859428 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.6801275605604229 = (6721.523183882363 - 2150.030017578125) / 6721.523183882363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Ctor4_ActivatorCreateInstance

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 57.4415398964921 (T) = (0 -9989.64144561012) / Math.Sqrt((75138.77160699888 / (33)) + (29933.13385687595 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.4302230939743098 = (17532.548862483563 - 9989.64144561012) / 17532.548862483563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod4_int_string_struct_class

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 100.70754553063318 (T) = (0 -1504.280227083333) / Math.Sqrt((27452.9322340278 / (33)) + (911.4300408168743 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.7060812888514478 = (5118.014505456377 - 1504.280227083333) / 5118.014505456377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Ctor0_ActivatorCreateInstance_NoParams

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 107.55946343248478 (T) = (0 -2042.67862330443) / Math.Sqrt((8206.305639041591 / (32)) + (552.1076960636332 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.5485512509220376 = (4524.718758167767 - 2042.67862330443) / 4524.718758167767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Field_Get_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 104.2770803159117 (T) = (0 -2225.897859906463) / Math.Sqrt((6662.075341681476 / (33)) + (92.90292664392777 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.424707062859651 = (3869.1555487729393 - 2225.897859906463) / 3869.1555487729393 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 13, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseGeneric - Duration of single invocation 2.59 μs 2.00 μs 0.77 0.16 False
Parse - Duration of single invocation 5.70 μs 4.92 μs 0.86 0.05 False
TryParseGeneric - Duration of single invocation 5.61 μs 4.97 μs 0.89 0.08 False
Parse - Duration of single invocation 2.74 μs 2.00 μs 0.73 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.TryParseGeneric(text: "Red")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 9.092886807156567 (T) = (0 -2024.2273084861304) / Math.Sqrt((194968.81550947228 / (33)) + (876.0439600340983 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2635404050769014 = (2748.5924855083204 - 2024.2273084861304) / 2748.5924855083204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.Parse(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 12.79756005021258 (T) = (0 -4982.648352560271) / Math.Sqrt((9136.776649195343 / (33)) + (6850.339940114233 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.1351533232746459 = (5761.3083181709335 - 4982.648352560271) / 5761.3083181709335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 9.111451618381873 (T) = (0 -4917.241656755408) / Math.Sqrt((189096.6362154182 / (33)) + (6430.371472250081 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.14911941711142299 = (5779.003253385231 - 4917.241656755408) / 5779.003253385231 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.Parse(text: "Red")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 19.380926122463073 (T) = (0 -1983.5500253315931) / Math.Sqrt((35574.775183504986 / (32)) + (496.17249383706763 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.264869971932459 = (2698.2301764298927 - 1983.5500253315931) / 2698.2301764298927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in System.Reflection.Activator<EmptyClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateInstanceGeneric - Duration of single invocation 1.41 μs 828.63 ns 0.59 0.14 False
CreateInstanceType - Duration of single invocation 4.51 μs 2.01 μs 0.45 0.05 False
CreateInstanceNames - Duration of single invocation 39.85 μs 33.80 μs 0.85 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyClass&gt;*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Activator<EmptyClass>.CreateInstanceGeneric

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 55.641262225724454 (T) = (0 -836.6483326342113) / Math.Sqrt((1689.8309589503967 / (33)) + (128.48843194078864 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.4167681428943528 = (1434.5038297224908 - 836.6483326342113) / 1434.5038297224908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Activator&lt;EmptyClass&gt;.CreateInstanceType

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 72.3936058928509 (T) = (0 -2037.954521383852) / Math.Sqrt((8477.293761124443 / (33)) + (1794.5791959100889 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.5468589872934412 = (4497.395875097214 - 2037.954521383852) / 4497.395875097214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Activator&lt;EmptyClass&gt;.CreateInstanceNames

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 55.54331422829809 (T) = (0 -33788.22183077119) / Math.Sqrt((451367.4745868173 / (33)) + (482.4261058711003 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.16243801295467827 = (40341.15964355825 - 33788.22183077119) / 40341.15964355825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ScopeValidation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TransientWithScopeValidation - Duration of single invocation 6.62 μs 4.49 μs 0.68 0.08 False
Transient - Duration of single invocation 6.27 μs 4.29 μs 0.68 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 24.967186220072062 (T) = (0 -4575.219810955246) / Math.Sqrt((22140.485032628367 / (32)) + (13070.980546153161 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.31690505446457384 = (6697.78021468023 - 4575.219810955246) / 6697.78021468023 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 36.15171962444827 (T) = (0 -4337.372969411922) / Math.Sqrt((26156.362109530266 / (33)) + (5383.2051007258015 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.32975354643569854 = (6471.310584854601 - 4337.372969411922) / 6471.310584854601 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 45.63 μs 41.56 μs 0.91 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<LoginViewModel>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 28.432522976988054 (T) = (0 -41433.446031207844) / Math.Sqrt((277802.0098685331 / (33)) + (31306.370388114232 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.09622253186235474 = (45844.74330455153 - 41433.446031207844) / 45844.74330455153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 9.68 ms 8.88 ms 0.92 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 5.343388313559283 (T) = (0 -8987242.962049637) / Math.Sqrt((18105398665.639378 / (33)) + (21426928224.694405 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.05935096693820525 = (9554299.89950272 - 8987242.962049637) / 9554299.89950272 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 13, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetConverter - Duration of single invocation 2.70 μs 2.15 μs 0.79 0.03 False
GetConverter - Duration of single invocation 7.63 μs 4.47 μs 0.59 0.05 False
GetConverter - Duration of single invocation 4.40 μs 2.97 μs 0.68 0.03 False
GetConverter - Duration of single invocation 7.45 μs 4.63 μs 0.62 0.08 False
GetConverter - Duration of single invocation 4.41 μs 2.98 μs 0.68 0.03 False
GetConverter - Duration of single invocation 7.67 μs 4.53 μs 0.59 0.12 False
GetConverter - Duration of single invocation 7.93 μs 5.17 μs 0.65 0.03 False
GetConverter - Duration of single invocation 11.20 μs 6.03 μs 0.54 0.11 False
GetConverter - Duration of single invocation 7.68 μs 4.55 μs 0.59 0.05 False
GetConverter - Duration of single invocation 7.98 μs 4.54 μs 0.57 0.11 False
GetConverter - Duration of single invocation 4.60 μs 2.97 μs 0.65 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'

Payloads

Baseline
Compare

Histogram

System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.IDerived))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 52.43149756909497 (T) = (0 -2153.906526982806) / Math.Sqrt((1469.6486305307812 / (33)) + (120.48959542044364 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.19947140161987525 = (2690.605346693736 - 2153.906526982806) / 2690.605346693736 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Enum))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 45.205708560128734 (T) = (0 -4533.789516539355) / Math.Sqrt((12845.167146442061 / (33)) + (7766.410516655079 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3945757405209279 = (7488.6155378715475 - 4533.789516539355) / 7488.6155378715475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassWithNoConverter))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 68.16613672456663 (T) = (0 -2993.3926505859113) / Math.Sqrt((3872.034758839753 / (33)) + (725.095757681594 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.33282304027002246 = (4486.654712712814 - 2993.3926505859113) / 4486.654712712814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Guid))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 69.81018522822957 (T) = (0 -4661.068354160732) / Math.Sqrt((26124.994089485794 / (33)) + (1769.1339986648748 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3801132064084882 = (7519.225126825733 - 4661.068354160732) / 7519.225126825733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassIDerived))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 55.980919379225355 (T) = (0 -3004.201041105116) / Math.Sqrt((6316.300627196015 / (33)) + (1083.7227922605148 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3353685001702835 = (4520.100298999994 - 3004.201041105116) / 4520.100298999994 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 67.12349625919501 (T) = (0 -4567.536975732172) / Math.Sqrt((10789.698884209563 / (22)) + (2930.8590865880756 / (2))) is greater than 2.0738730678945014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (2) - 2, .975) and 0.39391145532765104 = (7536.088605999606 - 4567.536975732172) / 7536.088605999606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.DerivedClass))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 39.76564763764994 (T) = (0 -5098.6323107546505) / Math.Sqrt((15993.073871040615 / (33)) + (9145.097614020551 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.35676305317116563 = (7926.522778100616 - 5098.6323107546505) / 7926.522778100616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeEnum))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 58.5842485221714 (T) = (0 -6095.8904748328405) / Math.Sqrt((43796.894549507066 / (33)) + (8824.002105114745 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.42131720202270856 = (10534.079285128593 - 6095.8904748328405) / 10534.079285128593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int?))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 21.673904914284876 (T) = (0 -4686.455242975822) / Math.Sqrt((13373.669258777323 / (33)) + (36101.516276844166 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3858596456976592 = (7630.918909895773 - 4686.455242975822) / 7630.918909895773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeValueType?))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 72.72074768740121 (T) = (0 -4567.502091915883) / Math.Sqrt((29946.449824317777 / (33)) + (1770.2549001665868 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.402662135146482 = (7646.429869360362 - 4567.502091915883) / 7646.429869360362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(string))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 40.86761956296033 (T) = (0 -3002.7376570854394) / Math.Sqrt((8148.310317495331 / (32)) + (2224.7906183778205 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.3347578314155848 = (4513.751230585754 - 3002.7376570854394) / 4513.751230585754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 40.32 ms 27.63 ms 0.69 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<CollectionsOfPrimitives>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 22.471269055939665 (T) = (0 -27080148.68148148) / Math.Sqrt((1332192501817.0527 / (29)) + (604983834182.0679 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.32877673125139506 = (40344472.46140372 - 27080148.68148148) / 40344472.46140372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 100.32 μs 84.74 μs 0.84 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<Location>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 24.54442709184628 (T) = (0 -85294.69808269336) / Math.Sqrt((1332788.9581871345 / (33)) + (615928.1996001985 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.1451821940489222 = (99781.14340727113 - 85294.69808269336) / 99781.14340727113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Transient - Duration of single invocation 6.23 μs 4.24 μs 0.68 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.Transient

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 16.12061964983695 (T) = (0 -4368.8174987646125) / Math.Sqrt((18292.481056897876 / (33)) + (31993.129618136587 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.32190058938538535 = (6442.738970683975 - 4368.8174987646125) / 6442.738970683975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NewCustomConverter - Duration of single invocation 1.03 ms 714.83 μs 0.70 0.14 False
NewJsonSerializerContext - Duration of single invocation 241.83 μs 191.75 μs 0.79 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverter

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 55.35430238824377 (T) = (0 -716483.0860795456) / Math.Sqrt((935232736.0470915 / (33)) + (5446503.228736139 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.3009914128176998 = (1024998.9760035495 - 716483.0860795456) / 1024998.9760035495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;.NewJsonSerializerContext

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 10.71957265893956 (T) = (0 -187337.70595660128) / Math.Sqrt((124356519.88530575 / (13)) + (38988195.45837024 / (2))) is greater than 2.160368656462814 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (2) - 2, .975) and 0.23574323991795124 = (245124.04173760812 - 187337.70595660128) / 245124.04173760812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in System.Reflection.Activator<EmptyStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateInstanceType - Duration of single invocation 5.16 μs 2.02 μs 0.39 0.24 False
CreateInstanceGeneric - Duration of single invocation 2.28 μs 896.98 ns 0.39 0.05 False
CreateInstanceNames - Duration of single invocation 35.20 μs 26.91 μs 0.76 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Activator<EmptyStruct>.CreateInstanceType

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 143.10870318822433 (T) = (0 -2022.7727618805188) / Math.Sqrt((16683.557604996622 / (33)) + (59.58061367134751 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.6207759066629737 = (5333.977448745136 - 2022.7727618805188) / 5333.977448745136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Activator&lt;EmptyStruct&gt;.CreateInstanceGeneric

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 104.35107513891339 (T) = (0 -908.1652009466075) / Math.Sqrt((1447.6929797240473 / (33)) + (250.030268590861 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.5989140873896428 = (2264.2660148197783 - 908.1652009466075) / 2264.2660148197783 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Activator&lt;EmptyStruct&gt;.CreateInstanceNames

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 11.446309270650191 (T) = (0 -27595.316811149878) / Math.Sqrt((342386.18132343685 / (33)) + (929195.6221319919 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2223148318256127 = (35483.91809494036 - 27595.316811149878) / 35483.91809494036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ServiceProvider - Duration of single invocation 6.10 μs 2.91 μs 0.48 0.12 False
CreateInstance - Duration of single invocation 26.71 μs 21.42 μs 0.80 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 61.82034195425975 (T) = (0 -2942.5337429470023) / Math.Sqrt((16476.951963811658 / (22)) + (2628.203815434161 / (2))) is greater than 2.0738730678945014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (2) - 2, .975) and 0.48829653086093977 = (5750.466667536587 - 2942.5337429470023) / 5750.466667536587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 12.09650367405423 (T) = (0 -21705.176893137133) / Math.Sqrt((637437.5779765503 / (33)) + (165051.77147440804 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.1509973757890206 = (25565.500357915666 - 21705.176893137133) / 25565.500357915666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 13, 2022 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 34.78 ms 25.64 ms 0.74 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<CollectionsOfPrimitives>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 41.781125313601414 (T) = (0 -25809295.885952376) / Math.Sqrt((839397885146.7491 / (32)) + (56314154795.090965 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.2740653313284105 = (35553193.69604101 - 25809295.885952376) / 35553193.69604101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Singleton - Duration of single invocation 58.71 μs 51.53 μs 0.88 0.08 False
BuildProvider - Duration of single invocation 26.91 μs 21.97 μs 0.82 0.08 False
Scoped - Duration of single invocation 66.35 μs 54.91 μs 0.83 0.11 False
Transient - Duration of single invocation 57.63 μs 49.98 μs 0.87 0.14 False
Singleton - Duration of single invocation 61.86 μs 51.15 μs 0.83 0.15 False
Singleton - Duration of single invocation 58.38 μs 51.58 μs 0.88 0.19 False
Transient - Duration of single invocation 58.08 μs 49.81 μs 0.86 0.03 False
Scoped - Duration of single invocation 62.88 μs 55.19 μs 0.88 0.03 False
BuildProvider - Duration of single invocation 27.09 μs 22.03 μs 0.81 0.03 False
Scoped - Duration of single invocation 65.53 μs 55.83 μs 0.85 0.10 False
BuildProvider - Duration of single invocation 27.21 μs 22.08 μs 0.81 0.05 False
Singleton - Duration of single invocation 58.01 μs 50.70 μs 0.87 0.08 False
BuildProvider - Duration of single invocation 28.26 μs 22.04 μs 0.78 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 5.7280550663284435 (T) = (0 -52749.30146822907) / Math.Sqrt((2097891.882632373 / (22)) + (2951072.63590943 / (2))) is greater than 2.0738730678945014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (2) - 2, .975) and 0.11979723253484065 = (59928.57943418931 - 52749.30146822907) / 59928.57943418931 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 47.578249292198784 (T) = (0 -22015.373559272302) / Math.Sqrt((420034.27775120904 / (33)) + (3262.572886415101 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.20570143287405535 = (27716.748424879792 - 22015.373559272302) / 27716.748424879792 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 5.188242349584059 (T) = (0 -56385.1112742078) / Math.Sqrt((2096239.200001678 / (33)) + (4349751.651207649 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.12100685584503432 = (64147.384594693896 - 56385.1112742078) / 64147.384594693896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 8.511303588732046 (T) = (0 -50878.75161312452) / Math.Sqrt((2398752.8522201846 / (33)) + (1629930.581279287 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.13615064254691273 = (58897.71309563956 - 50878.75161312452) / 58897.71309563956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 7.3808145586252625 (T) = (0 -52134.690842416516) / Math.Sqrt((1670547.361721142 / (22)) + (1955130.6070667827 / (2))) is greater than 2.0738730678945014 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (2) - 2, .975) and 0.12687371953297347 = (59710.36722721276 - 52134.690842416516) / 59710.36722721276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 5.844754657398231 (T) = (0 -52760.77068059546) / Math.Sqrt((2697959.7994892974 / (33)) + (2769138.1303307773 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.118277529809355 = (59838.29658916097 - 52760.77068059546) / 59838.29658916097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 8.283088707700143 (T) = (0 -50726.16921160294) / Math.Sqrt((2708020.6977937357 / (33)) + (1696038.4577144603 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.13605286074271142 = (58714.436227210404 - 50726.16921160294) / 58714.436227210404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 8.892104609725253 (T) = (0 -56042.46578297876) / Math.Sqrt((1532284.4856771936 / (29)) + (1438686.371880066 / (2))) is greater than 2.045229642128227 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (2) - 2, .975) and 0.12236586202388448 = (63856.29655680501 - 56042.46578297876) / 63856.29655680501 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 65.70033787541732 (T) = (0 -22019.85073393259) / Math.Sqrt((222859.34433693546 / (33)) + (226.27633390416247 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.19822994000193805 = (27464.047153351945 - 22019.85073393259) / 27464.047153351945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 10.45835593129888 (T) = (0 -56484.19544597571) / Math.Sqrt((1793934.1057575324 / (27)) + (854691.2999178307 / (2))) is greater than 2.051830516474954 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (2) - 2, .975) and 0.11512942559780251 = (63833.28486669975 - 56484.19544597571) / 63833.28486669975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 49.27116334941157 (T) = (0 -22122.91481727347) / Math.Sqrt((338265.5230006365 / (33)) + (3682.61706186215 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.19672509659186432 = (27540.901282251372 - 22122.91481727347) / 27540.901282251372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 6.990452020703359 (T) = (0 -51771.884586139175) / Math.Sqrt((2182037.4120626566 / (32)) + (2309664.659014117 / (2))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (2) - 2, .975) and 0.1299229713980723 = (59502.64503515071 - 51771.884586139175) / 59502.64503515071 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 46.77017959063823 (T) = (0 -22061.60578480794) / Math.Sqrt((448604.9902043012 / (33)) + (1413.3311716498472 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.20224617403539047 = (27654.65368734772 - 22061.60578480794) / 27654.65368734772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 109.49 μs 76.90 μs 0.70 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<Location>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 24.969215009150595 (T) = (0 -78051.2983450164) / Math.Sqrt((2191097.4607627383 / (33)) + (2639952.273141074 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2736112349106949 = (107451.13649358338 - 78051.2983450164) / 107451.13649358338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ThrowAndCatch - Duration of single invocation 13.20 μs 12.25 μs 0.93 0.19 False
ThrowAndCatchDeep - Duration of single invocation 17.54 μs 16.10 μs 0.92 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'

Payloads

Baseline
Compare

Histogram

Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 7.804375487702078 (T) = (0 -12150.577963051215) / Math.Sqrt((102503.81794659355 / (33)) + (19611.61233272747 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.06802111647550343 = (13037.396209129714 - 12150.577963051215) / 13037.396209129714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionSoftware)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 5.125052083983643 (T) = (0 -16273.16696420799) / Math.Sqrt((104620.65015139937 / (33)) + (57220.72618839978 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.053159937410291784 = (17186.817084713493 - 16273.16696420799) / 17186.817084713493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 46.33 μs 35.44 μs 0.76 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<LoginViewModel>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 79.09006045912366 (T) = (0 -35539.90118597889) / Math.Sqrt((330711.75974110316 / (33)) + (20638.612953379627 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.24092219008863353 = (46819.839444560625 - 35539.90118597889) / 46819.839444560625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 5b8a981452b02e034857caa566f83ca1801000b7
Compare 6407eae2c9cda49c725e744f9628bca65b6e71d2
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 362.93 μs 270.77 μs 0.75 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<IndexViewModel>.BinaryFormatter_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsImprovementStdDev: Marked as improvement because 37.85006125258136 (T) = (0 -273092.4548592505) / Math.Sqrt((31487194.720570203 / (33)) + (10801818.071467739 / (2))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (2) - 2, .975) and 0.2589272582167732 = (368509.6475173466 - 273092.4548592505) / 368509.6475173466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Jan 10, 2023

looks likeprobably dotnet/runtime#78840

cc @vargaz @SamMonoRT

@kotlarmilos kotlarmilos mentioned this issue Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @kotlarmilos