Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 568 Regressions on 11/15/2022 10:02:59 PM #78824

Closed
performanceautofiler bot opened this issue Nov 22, 2022 · 33 comments
Closed

[Perf] Linux/x64: 568 Regressions on 11/15/2022 10:02:59 PM #78824

performanceautofiler bot opened this issue Nov 22, 2022 · 33 comments
Assignees
@kg
Labels
arch-x64 area-Codegen-meta-mono os-linux Linux OS (any supported distro)

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.Perf_Dictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ContainsValue - Duration of single invocation 438.63 ms 481.89 ms 1.10 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_Dictionary*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_Dictionary.ContainsValue(Items: 3000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 481.8899057857143 > 461.3351680875.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.69411309234247 (T) = (0 -484991048.3220324) / Math.Sqrt((64720213261170.91 / (19)) + (35990464111782.46 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.09159922293863344 = (444294057.8653171 - 484991048.3220324) / 444294057.8653171 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableQueue - Duration of single invocation 34.19 μs 38.34 μs 1.12 0.08 False
ConcurrentQueue - Duration of single invocation 38.43 μs 40.57 μs 1.06 0.09 False
ImmutableStack - Duration of single invocation 33.93 μs 38.82 μs 1.14 0.12 False
ImmutableSortedSet - Duration of single invocation 145.38 μs 155.82 μs 1.07 0.05 False
List - Duration of single invocation 12.80 μs 13.65 μs 1.07 0.11 False
HashSet - Duration of single invocation 15.94 μs 16.94 μs 1.06 0.09 False
ConcurrentStack - Duration of single invocation 23.53 μs 25.09 μs 1.07 0.11 False
SortedDictionary - Duration of single invocation 84.75 μs 91.73 μs 1.08 0.12 False
ImmutableHashSet - Duration of single invocation 277.53 μs 295.39 μs 1.06 0.04 False
LinkedList - Duration of single invocation 16.71 μs 18.65 μs 1.12 0.05 False
ImmutableArray - Duration of single invocation 10.78 μs 12.43 μs 1.15 0.12 False
ConcurrentBag - Duration of single invocation 25.18 μs 27.14 μs 1.08 0.05 False
ImmutableSortedDictionary - Duration of single invocation 144.15 μs 154.85 μs 1.07 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ImmutableQueue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.34330977839052 > 36.482979412625475.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.529600828038468 (T) = (0 -38862.16970030248) / Math.Sqrt((1179374.6513844314 / (19)) + (619915.1108580803 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12170007046443627 = (34645.776285109554 - 38862.16970030248) / 34645.776285109554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ConcurrentQueue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.572623433377316 > 39.90799484421611.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.22137582558428 (T) = (0 -41029.89881715364) / Math.Sqrt((169762.4838628088 / (19)) + (349044.2824229279 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07770028181105548 = (38071.715772592775 - 41029.89881715364) / 38071.715772592775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ImmutableStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.8194844656454 > 35.80043910995674.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.818735693100905 (T) = (0 -38162.81173982531) / Math.Sqrt((168186.67891967818 / (19)) + (612479.3041314126 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11708215051479993 = (34162.94112499983 - 38162.81173982531) / 34162.94112499983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.8202511607143 > 151.81342036553897.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.365500890154056 (T) = (0 -158103.43229361068) / Math.Sqrt((8837100.267035509 / (19)) + (5116330.932637106 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09088671621436457 = (144931.11882622153 - 158103.43229361068) / 144931.11882622153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.647552306884291 > 13.35496466281323.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.739220034106559 (T) = (0 -13891.2854813982) / Math.Sqrt((29390.195621404055 / (19)) + (209386.245305886 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08728993223153203 = (12776.063743079092 - 13891.2854813982) / 12776.063743079092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.HashSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.938743252564898 > 16.74611441397111.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.136518414175729 (T) = (0 -17090.100185238596) / Math.Sqrt((65247.77588590908 / (18)) + (202441.45904661625 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.06505556108650296 = (16046.205296373784 - 17090.100185238596) / 16046.205296373784 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ConcurrentStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.09377265541265 > 24.65082953617722.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.013367682458798 (T) = (0 -25561.556160270968) / Math.Sqrt((194803.1408084503 / (19)) + (1482097.012333907 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08554230635462624 = (23547.26850408029 - 25561.556160270968) / 23547.26850408029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.73196598837212 > 88.76401922641917.
IsChangePoint: Marked as a change because one of 10/2/2022 5:38:34 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.537176863918285 (T) = (0 -90845.18164414854) / Math.Sqrt((3598469.470098597 / (19)) + (2340549.1938484935 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08442993118004567 = (83772.29273383612 - 90845.18164414854) / 83772.29273383612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ImmutableHashSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 295.3904866898148 > 289.26546470774286.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.242167327385864 (T) = (0 -296497.4898029521) / Math.Sqrt((11635543.617334193 / (19)) + (37412636.473999776 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07730768062730398 = (275220.8075136947 - 296497.4898029521) / 275220.8075136947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.649057572238327 > 17.65429289169845.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/24/2022 2:09:58 PM, 10/25/2022 2:56:59 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.273663151568933 (T) = (0 -17870.93919956162) / Math.Sqrt((41752.74288955493 / (19)) + (47410.197343298925 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.060221317680615824 = (16855.85726445949 - 17870.93919956162) / 16855.85726445949 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.428585837323755 > 11.39026500207959.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.393562169084422 (T) = (0 -11560.35398294969) / Math.Sqrt((28740.36457251683 / (19)) + (71500.40371980885 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.060817755603776766 = (10897.587188640126 - 11560.35398294969) / 10897.587188640126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ConcurrentBag(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.1382285597826 > 26.4771623020809.
IsChangePoint: Marked as a change because one of 7/29/2022 12:14:05 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.058003498166956 (T) = (0 -27297.30104349707) / Math.Sqrt((311515.3013416861 / (19)) + (79671.61527424428 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0710703410179256 = (25486.002177554667 - 27297.30104349707) / 25486.002177554667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.85241879284084 > 151.03450311798454.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.908045521662164 (T) = (0 -156092.16346025694) / Math.Sqrt((5238059.258496159 / (19)) + (5085725.437286132 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07460460870926412 = (145255.4383213965 - 156092.16346025694) / 145255.4383213965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 51.15 ns 54.52 ns 1.07 0.11 False
TryParse - Duration of single invocation 158.93 ns 176.50 ns 1.11 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: 127)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.521831462956825 > 54.01611770847977.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.523778094809566 (T) = (0 -55.37641394709205) / Math.Sqrt((0.7388647577272073 / (19)) + (1.2636651989916332 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07015259747269702 = (51.74627812694243 - 55.37641394709205) / 51.74627812694243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_SByte.TryParse(value: "127")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 176.49808150235563 > 167.21859127254797.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.621156578716156 (T) = (0 -176.88396480032512) / Math.Sqrt((33.25892992315662 / (22)) + (54.679738065506605 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.09035225788377144 = (162.22643968622887 - 176.88396480032512) / 162.22643968622887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateActivityEventsLarge - Duration of single invocation 41.69 μs 47.54 μs 1.14 0.14 False
EnumerateActivityTagObjectsSmall - Duration of single invocation 282.79 ns 306.40 ns 1.08 0.11 False
EnumerateActivityEventsSmall - Duration of single invocation 280.28 ns 308.70 ns 1.10 0.11 False
EnumerateActivityLinkTagsSmall - Duration of single invocation 280.59 ns 313.95 ns 1.12 0.10 False
EnumerateActivityLinksLarge - Duration of single invocation 43.23 μs 49.26 μs 1.14 0.20 False
EnumerateActivityLinksSmall - Duration of single invocation 285.27 ns 307.65 ns 1.08 0.10 False
EnumerateActivityTagObjectsLarge - Duration of single invocation 41.63 μs 48.52 μs 1.17 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.53696032222986 > 43.91877348259876.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.103481507112924 (T) = (0 -47404.1314738906) / Math.Sqrt((393815.3939847913 / (19)) + (1428745.3865194048 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12815852574834075 = (42019.03402045918 - 47404.1314738906) / 42019.03402045918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.40332650488625 > 296.71939502645716.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.486327313361663 (T) = (0 -310.2627367660254) / Math.Sqrt((7.154585687153589 / (19)) + (36.05020567586991 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09809765073029739 = (282.5456702868666 - 310.2627367660254) / 282.5456702868666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 308.6993213112313 > 294.21723590949233.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.2252236783252 (T) = (0 -312.0569540062667) / Math.Sqrt((8.926279791897834 / (19)) + (29.254584099427316 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10563187868746193 = (282.24308652959746 - 312.0569540062667) / 282.24308652959746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 313.94724289267583 > 294.8251040373757.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.487855446311492 (T) = (0 -308.8799166368641) / Math.Sqrt((17.554113882762383 / (19)) + (45.076986144445065 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09904885399545543 = (281.04293591132875 - 308.8799166368641) / 281.04293591132875 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.25756502227463 > 46.057512315619505.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.679915295917407 (T) = (0 -47615.84418110454) / Math.Sqrt((1107100.0184642572 / (19)) + (1468990.7225825577 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10570849140490976 = (43063.65063779514 - 47615.84418110454) / 43063.65063779514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 307.64509566633154 > 298.75212654872865.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.865183483289318 (T) = (0 -316.5900807152343) / Math.Sqrt((48.123041562358075 / (19)) + (71.03512158600839 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10939800739396868 = (285.3710558385806 - 316.5900807152343) / 285.3710558385806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.520759048507465 > 43.759563508563794.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.648120066304891 (T) = (0 -46833.15565461156) / Math.Sqrt((810546.4760832222 / (19)) + (2588761.2445831182 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.11175582515868558 = (42125.39713738569 - 46833.15565461156) / 42125.39713738569 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BenchNeuralJagged - Duration of single invocation 6.04 secs 6.62 secs 1.10 0.06 False
BenchIDEAEncryption - Duration of single invocation 9.27 secs 10.41 secs 1.12 0.05 False
BenchBitOps - Duration of single invocation 4.38 secs 5.22 secs 1.19 0.07 False
BenchStringSort - Duration of single invocation 7.37 secs 8.00 secs 1.08 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchNeuralJagged

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.615823691785714 > 6.347369698237499.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.583473359891205 (T) = (0 -6479218693.893837) / Math.Sqrt((6399144640742993 / (19)) + (10480998240623520 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06385450530702425 = (6090324063.650001 - 6479218693.893837) / 6090324063.650001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchIDEAEncryption

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.413443517 > 9.787536256200001.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.591077121631784 (T) = (0 -10244298498.85224) / Math.Sqrt((34139661418820336 / (19)) + (43172912296798070 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10324104147593277 = (9285639414.889116 - 10244298498.85224) / 9285639414.889116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchBitOps

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.224245457733334 > 4.5108891697799995.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 11/17/2022 4:58:45 AM, 11/18/2022 7:31:41 AM, 11/19/2022 12:25:45 PM, 11/21/2022 12:46:21 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.170850108790825 (T) = (0 -5132890486.533333) / Math.Sqrt((97752812894802400 / (45)) + (16691461525905764 / (2))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (2) - 2, .025) and -0.14063617394258876 = (4500024288.017789 - 5132890486.533333) / 4500024288.017789 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchStringSort

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.000615645071429 > 7.795656629962501.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/15/2022 9:29:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.14574219937624 (T) = (0 -8094585780.40047) / Math.Sqrt((9470466035083816 / (19)) + (13095930579679076 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08837530419029949 = (7437311145.554212 - 8094585780.40047) / 7437311145.554212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Ctor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 178.26 ns 192.75 ns 1.08 0.05 False
Ctor - Duration of single invocation 175.91 ns 193.71 ns 1.10 0.09 False
Ctor - Duration of single invocation 178.34 ns 192.24 ns 1.08 0.09 False
Ctor - Duration of single invocation 178.41 ns 190.74 ns 1.07 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Ctor*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: True, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 192.7548833468192 > 187.35719322139352.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.68496563369653 (T) = (0 -192.7005969642407) / Math.Sqrt((2.5226498963335873 / (19)) + (12.67709371892038 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08014248179619249 = (178.40294240051986 - 192.7005969642407) / 178.40294240051986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 193.7130732241908 > 184.47398342598805.
IsChangePoint: Marked as a change because one of 8/5/2022 8:01:14 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.566375975213855 (T) = (0 -190.68281878581715) / Math.Sqrt((3.8298040513475033 / (19)) + (14.662828946210235 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07471549275532789 = (177.42632359095285 - 190.68281878581715) / 177.42632359095285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 192.24497801309903 > 187.55733744908338.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.828506509649632 (T) = (0 -194.2242803384707) / Math.Sqrt((7.859587397310964 / (19)) + (8.237438457295374 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07862979591259572 = (180.0657473717787 - 194.2242803384707) / 180.0657473717787 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.7427701861213 > 186.8452876808159.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.934606406743549 (T) = (0 -192.80941568589952) / Math.Sqrt((35.975773624385 / (19)) + (7.836212254283477 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.06475011160450633 = (181.0841939197816 - 192.80941568589952) / 181.0841939197816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBoolean - Duration of single invocation 1.91 μs 2.06 μs 1.08 0.14 False
GetInt16 - Duration of single invocation 10.92 μs 12.90 μs 1.18 0.27 False
GetUInt32 - Duration of single invocation 10.36 μs 11.08 μs 1.07 0.18 False
GetInt32 - Duration of single invocation 10.84 μs 12.12 μs 1.12 0.17 False
GetDecimal - Duration of single invocation 64.22 μs 73.45 μs 1.14 0.04 False
GetSingle - Duration of single invocation 57.95 μs 64.53 μs 1.11 0.05 False
GetDouble - Duration of single invocation 58.38 μs 64.41 μs 1.10 0.04 False
GetByte - Duration of single invocation 10.69 μs 11.78 μs 1.10 0.19 False
GetInt64 - Duration of single invocation 10.42 μs 11.08 μs 1.06 0.19 False
GetSByte - Duration of single invocation 11.79 μs 13.76 μs 1.17 0.20 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetBoolean

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.062952790386961 > 2.00174178788088.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.5256645686751975 (T) = (0 -2129.044993229953) / Math.Sqrt((2075.7567359663876 / (19)) + (35358.48743572459 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10636090355219134 = (1924.367524552098 - 2129.044993229953) / 1924.367524552098 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetInt16

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.904447474385245 > 11.689036459736004.
IsChangePoint: Marked as a change because one of 10/1/2022 2:31:50 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.397981503815425 (T) = (0 -12092.56302274112) / Math.Sqrt((25939.13962503533 / (18)) + (126727.10863234501 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.09225017884772872 = (11071.239224239076 - 12092.56302274112) / 11071.239224239076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetUInt32

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.084582363541323 > 10.884948020256727.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.252891062911173 (T) = (0 -11315.22023462398) / Math.Sqrt((52066.56021667546 / (19)) + (154960.52436075284 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0710611404478613 = (10564.495160278666 - 11315.22023462398) / 10564.495160278666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetInt32

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.119705942661408 > 11.516908876782788.
IsChangePoint: Marked as a change because one of 8/2/2022 2:18:01 AM, 10/11/2022 5:13:36 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -4.555829903989289 (T) = (0 -11782.661325044206) / Math.Sqrt((182471.81309150145 / (19)) + (176319.8220126884 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05125587050769317 = (11208.176482623485 - 11782.661325044206) / 11208.176482623485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetDecimal

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.4465671923676 > 67.89705626515462.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.082617465244166 (T) = (0 -71196.82305530601) / Math.Sqrt((3247415.591247405 / (19)) + (1858328.7828009706 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09830922064502191 = (64824.02379677109 - 71196.82305530601) / 64824.02379677109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetSingle

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.53174823129251 > 62.03057445852027.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.068082899696465 (T) = (0 -64748.646579191925) / Math.Sqrt((2642995.617640975 / (22)) + (1293743.161622888 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.08374414832329964 = (59745.325203708766 - 64748.646579191925) / 59745.325203708766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetDouble

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.4057030357143 > 61.45778701770879.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.059034936519033 (T) = (0 -63201.466124677056) / Math.Sqrt((474007.0012541332 / (19)) + (1421284.1360059383 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.075169150180513 = (58782.8120943258 - 63201.466124677056) / 58782.8120943258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetByte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.778053590490051 > 11.233688856679088.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.2523414647011 (T) = (0 -11586.700081697476) / Math.Sqrt((48018.12989413711 / (19)) + (165068.9390922934 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06993552556843607 = (10829.34420327962 - 11586.700081697476) / 10829.34420327962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetInt64

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.079200044357703 > 10.974294703769052.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.774953472115598 (T) = (0 -11243.81679395996) / Math.Sqrt((25304.789412500904 / (19)) + (57443.38050958669 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06486579194226595 = (10558.90505549226 - 11243.81679395996) / 10558.90505549226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetSByte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.757354951908628 > 12.46906531882073.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.025638098114587 (T) = (0 -12955.22135716153) / Math.Sqrt((137197.95766298738 / (18)) + (189873.65489983468 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.07961215844330921 = (11999.884639907761 - 12955.22135716153) / 11999.884639907761 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Select - Duration of single invocation 5.83 μs 6.57 μs 1.13 0.09 False
Skip_One - Duration of single invocation 9.36 μs 10.06 μs 1.07 0.06 False
ToList - Duration of single invocation 7.48 μs 7.96 μs 1.06 0.03 False
OrderBy - Duration of single invocation 65.15 μs 70.98 μs 1.09 0.03 False
SequenceEqual - Duration of single invocation 87.99 ns 101.14 ns 1.15 0.23 False
WhereSingle_LastElementMatches - Duration of single invocation 4.57 μs 4.96 μs 1.09 0.17 False
Average - Duration of single invocation 4.44 μs 4.72 μs 1.06 0.06 False
SelectToList - Duration of single invocation 4.55 μs 5.03 μs 1.11 0.26 False
SelectToArray - Duration of single invocation 8.89 μs 9.71 μs 1.09 0.05 False
Sum - Duration of single invocation 4.22 μs 4.57 μs 1.08 0.09 False
GroupBy - Duration of single invocation 35.93 μs 38.17 μs 1.06 0.01 False
Select - Duration of single invocation 9.00 μs 10.83 μs 1.20 0.09 False
SelectToArray - Duration of single invocation 3.37 μs 3.75 μs 1.11 0.27 False
WhereSingle_LastElementMatches - Duration of single invocation 6.31 μs 6.76 μs 1.07 0.12 False
SelectToArray - Duration of single invocation 1.93 μs 2.06 μs 1.07 0.22 False
LastWithPredicate_FirstElementMatches - Duration of single invocation 79.79 ns 94.69 ns 1.19 0.20 False
WhereAny_LastElementMatches - Duration of single invocation 4.47 μs 4.91 μs 1.10 0.18 False
Select - Duration of single invocation 7.40 μs 8.13 μs 1.10 0.13 False
SingleWithPredicate_LastElementMatches - Duration of single invocation 5.52 μs 6.13 μs 1.11 0.13 False
AnyWithPredicate_LastElementMatches - Duration of single invocation 5.83 μs 6.15 μs 1.06 0.15 False
OrderByDescending - Duration of single invocation 90.69 μs 98.99 μs 1.09 0.02 False
WhereSingleOrDefault_LastElementMatches - Duration of single invocation 4.57 μs 4.85 μs 1.06 0.13 False
ElementAt - Duration of single invocation 1.37 μs 1.46 μs 1.06 0.13 False
AnyWithPredicate_LastElementMatches - Duration of single invocation 5.49 μs 5.94 μs 1.08 0.13 False
Range - Duration of single invocation 3.90 μs 4.29 μs 1.10 0.07 False
Zip - Duration of single invocation 14.14 μs 15.72 μs 1.11 0.15 False
Where - Duration of single invocation 7.82 μs 8.56 μs 1.09 0.12 False
WhereSelect - Duration of single invocation 9.19 μs 9.94 μs 1.08 0.14 False
SelectToArray - Duration of single invocation 3.40 μs 3.80 μs 1.12 0.32 False
Max - Duration of single invocation 4.44 μs 4.73 μs 1.06 0.07 False
AppendPrepend - Duration of single invocation 28.37 μs 30.48 μs 1.07 0.02 False
AnyWithPredicate_LastElementMatches - Duration of single invocation 60.85 μs 66.14 μs 1.09 0.02 False
Where - Duration of single invocation 9.52 μs 10.46 μs 1.10 0.11 False
SkipHalfTakeHalf - Duration of single invocation 6.51 μs 7.06 μs 1.08 0.10 False
Count - Duration of single invocation 2.42 μs 2.59 μs 1.07 0.12 False
SequenceEqual - Duration of single invocation 18.10 μs 20.30 μs 1.12 0.12 False
Contains_ElementNotFound - Duration of single invocation 13.34 μs 14.57 μs 1.09 0.08 False
OrderByThenBy - Duration of single invocation 65.46 μs 70.86 μs 1.08 0.02 False
Repeat - Duration of single invocation 3.87 μs 4.20 μs 1.09 0.11 False
CastToSameType - Duration of single invocation 4.33 μs 4.66 μs 1.08 0.14 False
Select - Duration of single invocation 9.06 μs 10.17 μs 1.12 0.11 False
FirstWithPredicate_LastElementMatches - Duration of single invocation 60.65 μs 67.44 μs 1.11 0.03 False
LastWithPredicate_FirstElementMatches - Duration of single invocation 17.94 μs 20.68 μs 1.15 0.10 False
EmptyTakeSelectToArray - Duration of single invocation 122.62 ns 136.42 ns 1.11 0.12 False
FirstWithPredicate_LastElementMatches - Duration of single invocation 5.53 μs 5.86 μs 1.06 0.13 False
WhereSelect - Duration of single invocation 11.01 μs 11.79 μs 1.07 0.09 False
WhereAny_LastElementMatches - Duration of single invocation 6.22 μs 6.71 μs 1.08 0.12 False
TakeLastHalf - Duration of single invocation 6.64 μs 7.19 μs 1.08 0.11 False
WhereSelect - Duration of single invocation 7.15 μs 7.75 μs 1.08 0.13 False
Take_All - Duration of single invocation 9.81 μs 10.72 μs 1.09 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Select(input: Array)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.573899380137399 > 6.148670657794804.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.610618950160893 (T) = (0 -6626.9315499411905) / Math.Sqrt((48171.94637641861 / (19)) + (57168.66856135792 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12148325067591412 = (5909.077595182239 - 6626.9315499411905) / 5909.077595182239 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.059923793010753 > 9.9977189659558.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.815848366836352 (T) = (0 -10154.394573186097) / Math.Sqrt((16347.548338699504 / (19)) + (55424.23312944503 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06610873819828181 = (9524.726896382981 - 10154.394573186097) / 9524.726896382981 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.960651919475655 > 7.82723401578387.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/5/2022 3:48:29 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.08249533745583 (T) = (0 -8018.963587601863) / Math.Sqrt((6088.041816225831 / (19)) + (15192.332465950527 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0709436953129352 = (7487.754606238828 - 8018.963587601863) / 7487.754606238828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.97652486667744 > 68.43290818600137.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.10165676394305 (T) = (0 -71721.26076647479) / Math.Sqrt((643072.8697432365 / (18)) + (2176911.0755882924 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.09727300937613104 = (65363.186876575826 - 71721.26076647479) / 65363.186876575826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: Array, input2: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.13864141311781 > 92.10651004320644.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.063884999372116 (T) = (0 -96.48114440941495) / Math.Sqrt((9.32638177076678 / (19)) + (8.296289091318268 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08009289881950575 = (89.32670931811941 - 96.48114440941495) / 89.32670931811941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.962902885010736 > 4.802337778161983.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -4.4120939527380925 (T) = (0 -4919.587459309564) / Math.Sqrt((59850.80932182498 / (19)) + (10259.990519124236 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05616986055323728 = (4657.951001113222 - 4919.587459309564) / 4657.951001113222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.715923293196165 > 4.661661611092582.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/5/2022 3:48:29 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.762557643240683 (T) = (0 -4765.799147305874) / Math.Sqrt((5666.327086671035 / (19)) + (6822.175888898935 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06648228507551422 = (4468.709151571536 - 4765.799147305874) / 4468.709151571536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.030092135618803 > 4.795170828730473.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.641170572173838 (T) = (0 -5137.554420873406) / Math.Sqrt((21800.707699545634 / (19)) + (47399.01227684585 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09846426478738962 = (4677.033732970633 - 5137.554420873406) / 4677.033732970633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.708020862855372 > 9.355116181925386.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 11/15/2022 9:22:46 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.00229910579522 (T) = (0 -9563.293188906073) / Math.Sqrt((14314.745446288638 / (16)) + (21922.368698738017 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.06224839801268814 = (9002.878429186243 - 9563.293188906073) / 9002.878429186243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.5735353358004955 > 4.441442359396665.
IsChangePoint: Marked as a change because one of 10/5/2022 3:48:29 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -4.413120954963827 (T) = (0 -4599.198701249565) / Math.Sqrt((73956.8851777503 / (19)) + (3092.4027043992046 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06463255826614417 = (4319.98689645543 - 4599.198701249565) / 4319.98689645543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.17104832644265 > 37.46442626035356.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/23/2022 8:43:59 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.309531172221597 (T) = (0 -38544.35188028105) / Math.Sqrt((51936.70410533846 / (19)) + (288052.52226568974 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07754909868703319 = (35770.39034903039 - 38544.35188028105) / 35770.39034903039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Select(input: IList)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.827675038091892 > 9.572979235949846.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.521668035246453 (T) = (0 -10194.861061013975) / Math.Sqrt((16616.648444464503 / (19)) + (73238.59048419495 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11301086958793095 = (9159.713835309092 - 10194.861061013975) / 9159.713835309092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.748709181177632 > 3.5195050970366077.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.428225482224596 (T) = (0 -3697.488936135353) / Math.Sqrt((43521.599989521084 / (19)) + (6784.756339024655 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09588730929881895 = (3373.9682034470457 - 3697.488936135353) / 3373.9682034470457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.760758682667248 > 6.6677607950157025.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.1985620659735226 (T) = (0 -6878.752023516853) / Math.Sqrt((73499.14071076369 / (19)) + (26582.741989832422 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06671971445200563 = (6448.509322854879 - 6878.752023516853) / 6448.509322854879 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.063284454424484 > 2.0282815517371118.
IsChangePoint: Marked as a change because one of 8/1/2022 11:22:56 PM, 9/30/2022 8:27:04 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.088694799059358 (T) = (0 -2120.711032419525) / Math.Sqrt((6870.160574363446 / (19)) + (7842.274017158825 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07843152954731948 = (1966.4772165087857 - 2120.711032419525) / 1966.4772165087857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 94.68602926823533 > 84.7707754513776.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.549859869472703 (T) = (0 -93.36440047411905) / Math.Sqrt((7.536004248630097 / (19)) + (5.4597046109155585 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.146927471761743 = (81.40392725156914 - 93.36440047411905) / 81.40392725156914 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.912705636248815 > 4.696495137571839.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.429114895724465 (T) = (0 -4860.140576079167) / Math.Sqrt((6062.328062260214 / (19)) + (21284.765927438293 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07689978096961635 = (4513.085304654074 - 4860.140576079167) / 4513.085304654074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Select(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.129366227457338 > 7.917130146676757.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.688917032090794 (T) = (0 -8158.150177786331) / Math.Sqrt((160506.33061796223 / (19)) + (15225.02430568751 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07080487679711794 = (7618.708463663479 - 8158.150177786331) / 7618.708463663479 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.12791428810384 > 5.805785366257189.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/6/2022 2:13:23 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.160017156500563 (T) = (0 -5977.012899369086) / Math.Sqrt((48005.525556848945 / (19)) + (39676.40892187891 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06919969014965215 = (5590.174552456618 - 5977.012899369086) / 5590.174552456618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.153323176539482 > 6.130679748054377.
IsChangePoint: Marked as a change because one of 10/7/2022 11:47:58 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.198250472894221 (T) = (0 -6242.396027980157) / Math.Sqrt((21114.55852826836 / (19)) + (19695.29506809141 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05926183935751877 = (5893.1567210675685 - 6242.396027980157) / 5893.1567210675685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.99195725022605 > 95.19298636651011.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/17/2022 12:47:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.60010816232941 (T) = (0 -99557.36903765089) / Math.Sqrt((1131348.8051775058 / (19)) + (2038841.9206319002 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09438187543709903 = (90971.32479271681 - 99557.36903765089) / 90971.32479271681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.85033684597384 > 4.807535172191619.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.360375546929633 (T) = (0 -4913.541326837479) / Math.Sqrt((9739.832038305216 / (19)) + (17049.15795193591 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06814476990567135 = (4600.0705758933755 - 4913.541326837479) / 4600.0705758933755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4554039166956068 > 1.4271988008947996.
IsChangePoint: Marked as a change because one of 7/23/2022 5:59:35 PM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.828141254744567 (T) = (0 -1465.9705928589751) / Math.Sqrt((617.0221575226662 / (19)) + (386.4934473877711 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07927277336156159 = (1358.2947972392406 - 1465.9705928589751) / 1358.2947972392406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.939360280067441 > 5.791303370610127.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.85251756244308 (T) = (0 -5960.423944745206) / Math.Sqrt((23009.377416306288 / (19)) + (37250.82953887599 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0711402144132588 = (5564.559956336027 - 5960.423944745206) / 5564.559956336027 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Range

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.291880557176484 > 4.091104611421682.
IsChangePoint: Marked as a change because one of 8/2/2022 9:59:24 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.21260061894788 (T) = (0 -4193.594791351582) / Math.Sqrt((5957.710796334189 / (19)) + (3341.889265302693 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07015364756236908 = (3918.6847616731384 - 4193.594791351582) / 3918.6847616731384 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.717554502012069 > 14.711297121502257.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.948792607830024 (T) = (0 -15807.03622906402) / Math.Sqrt((159869.38634507387 / (19)) + (286351.5723087817 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11508430962959425 = (14175.642229523219 - 15807.03622906402) / 14175.642229523219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.558041500667368 > 8.082924553074038.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.812793254759642 (T) = (0 -8477.707738938612) / Math.Sqrt((16006.630875036024 / (19)) + (34105.56384003785 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08614578149759691 = (7805.312954628797 - 8477.707738938612) / 7805.312954628797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.943856769173573 > 9.587300937017654.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.472501823351617 (T) = (0 -10072.802485737147) / Math.Sqrt((13824.757844131505 / (19)) + (37668.51342048449 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09651080148814228 = (9186.23188396022 - 10072.802485737147) / 9186.23188396022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.796552603275483 > 3.579088285249621.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.323537112147326 (T) = (0 -3861.5680818033384) / Math.Sqrt((16716.614877546177 / (19)) + (11612.043842426521 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11795005948426801 = (3454.150790586079 - 3861.5680818033384) / 3454.150790586079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.7261017624388355 > 4.554505049723881.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.840594731796683 (T) = (0 -4748.167180773046) / Math.Sqrt((7472.4684985483345 / (19)) + (3432.692890760527 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08198670461000089 = (4388.378489811952 - 4748.167180773046) / 4388.378489811952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.47532789026222 > 29.816290476712812.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.830226826905577 (T) = (0 -30254.770545239815) / Math.Sqrt((142573.55102802833 / (19)) + (148138.6080995764 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06714799068250996 = (28351.054220595906 - 30254.770545239815) / 28351.054220595906 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.14006155063291 > 63.513740810810816.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.790794995822438 (T) = (0 -66806.9640790155) / Math.Sqrt((620839.4307123251 / (19)) + (995190.8854185963 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10255006904008244 = (60593.13400359215 - 66806.9640790155) / 60593.13400359215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.462868272589937 > 10.054957220355247.
IsChangePoint: Marked as a change because one of 10/12/2022 1:08:53 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.05627783216855 (T) = (0 -10348.327811475774) / Math.Sqrt((15992.083284986144 / (19)) + (22189.468998887896 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07588810804583929 = (9618.405235718874 - 10348.327811475774) / 9618.405235718874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.063893286167595 > 6.819894598506816.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.996257472356136 (T) = (0 -7130.3935155899435) / Math.Sqrt((24625.581992070733 / (19)) + (30132.150479657077 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.09007071300481828 = (6541.221069901752 - 7130.3935155899435) / 6541.221069901752 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5917681566036688 > 2.53887825210194.
IsChangePoint: Marked as a change because one of 9/30/2022 12:37:41 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.144027043788558 (T) = (0 -2619.0330226492806) / Math.Sqrt((813.5103182664175 / (19)) + (5225.858052512091 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08227605293603171 = (2419.930678078192 - 2619.0330226492806) / 2419.930678078192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.303787868589744 > 19.274771932499313.
IsChangePoint: Marked as a change because one of 7/26/2022 11:31:56 PM, 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.282703718983454 (T) = (0 -20224.83892264077) / Math.Sqrt((169094.4966797505 / (19)) + (335859.7959603599 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10489119374961606 = (18304.8240741287 - 20224.83892264077) / 18304.8240741287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.569905397890356 > 14.020194946721672.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/23/2022 8:43:59 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.259387538376792 (T) = (0 -14771.876020333182) / Math.Sqrt((186740.72752196272 / (19)) + (110620.91958232444 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08876283680742249 = (13567.57920177431 - 14771.876020333182) / 13567.57920177431 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.85831640122265 > 67.96785831385992.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.890010326234826 (T) = (0 -71443.66741466282) / Math.Sqrt((1361397.6800607825 / (19)) + (618219.5923602215 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0931761707991431 = (65354.212178294605 - 71443.66741466282) / 65354.212178294605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Repeat

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.204210484989787 > 4.013535676933469.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.346968931699339 (T) = (0 -4239.908253881355) / Math.Sqrt((23101.30295714461 / (19)) + (4104.1784941734595 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09899429218458837 = (3857.9893308210335 - 4239.908253881355) / 3857.9893308210335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.65817101785167 > 4.53594117532858.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.286721418613114 (T) = (0 -4719.279428793277) / Math.Sqrt((21506.32415300666 / (19)) + (11381.867702354155 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08363861479807451 = (4355.030694132904 - 4719.279428793277) / 4355.030694132904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.170648915270501 > 9.564146204175072.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.828374257864434 (T) = (0 -10167.585995185642) / Math.Sqrt((21861.335978371615 / (19)) + (122288.15281033482 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.10507825272340006 = (9200.78371837309 - 10167.585995185642) / 9200.78371837309 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.43635766882184 > 63.58499387305682.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/17/2022 12:47:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.98535497612331 (T) = (0 -67118.31093104399) / Math.Sqrt((501412.6430385646 / (19)) + (2220353.761007721 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10717704058022089 = (60621.11881932663 - 67118.31093104399) / 60621.11881932663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IOrderedEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.68456561557671 > 19.417576730661906.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.03968766977419 (T) = (0 -20433.32875303082) / Math.Sqrt((107315.80476654321 / (19)) + (176088.27625183202 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1067555386454213 = (18462.368643792423 - 20433.32875303082) / 18462.368643792423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 136.4164824526043 > 125.40803429180376.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.32985984158867 (T) = (0 -134.14148322719663) / Math.Sqrt((6.444973650860975 / (19)) + (7.479113746028618 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11184806103975653 = (120.64731497732954 - 134.14148322719663) / 120.64731497732954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.863333887636227 > 5.784517592662089.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/6/2022 8:38:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.992275830289957 (T) = (0 -5972.101388650465) / Math.Sqrt((11613.419944532528 / (19)) + (16442.45419289411 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08144711112575433 = (5522.324048222465 - 5972.101388650465) / 5522.324048222465 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.794761488297498 > 11.635300503667601.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.258378396586862 (T) = (0 -11989.690141417774) / Math.Sqrt((60568.59407162334 / (19)) + (108279.61927583774 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0693580851847712 = (11212.044223096804 - 11989.690141417774) / 11212.044223096804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.712414689342743 > 6.608146679763328.
IsChangePoint: Marked as a change because one of 10/6/2022 12:41:43 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.790920144387803 (T) = (0 -6835.441732903843) / Math.Sqrt((8141.3835505737725 / (19)) + (69061.78119843478 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.083504708420684 = (6308.640543765777 - 6835.441732903843) / 6308.640543765777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.190021572580645 > 6.942723504555526.
IsChangePoint: Marked as a change because one of 10/10/2022 3:25:05 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.001203333789466 (T) = (0 -7201.2605339251595) / Math.Sqrt((10777.49375990279 / (19)) + (38320.249992986326 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09354619052030586 = (6585.236724658903 - 7201.2605339251595) / 6585.236724658903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.750340194118692 > 7.5873127126908715.
IsChangePoint: Marked as a change because one of 10/7/2022 6:01:23 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.09718970822469 (T) = (0 -7732.908398517199) / Math.Sqrt((10561.136213769147 / (19)) + (5383.19988583859 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06838784196013259 = (7237.9224985656065 - 7732.908398517199) / 7237.9224985656065 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.719265633632597 > 10.337742462431756.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.106998579219656 (T) = (0 -10823.566170300717) / Math.Sqrt((21027.67909450546 / (19)) + (24231.379454682225 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08991606527906133 = (9930.641922898401 - 10823.566170300717) / 9930.641922898401 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPinnableReference - Duration of single invocation 18.46 ns 22.48 ns 1.22 0.17 False
LastIndexOfAnyValues - Duration of single invocation 3.73 μs 4.02 μs 1.08 0.10 False
SequenceCompareTo - Duration of single invocation 1.13 μs 1.23 μs 1.09 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.GetPinnableReference(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.479669876839065 > 19.33436203673896.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.795591198047454 (T) = (0 -22.868952504535507) / Math.Sqrt((0.08413093665415962 / (19)) + (0.9576716921401349 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.23049886309479362 = (18.585106569719567 - 22.868952504535507) / 18.585106569719567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.021179324894515 > 3.923364347941656.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.2708319449180445 (T) = (0 -4071.1919802770826) / Math.Sqrt((23129.442389903168 / (19)) + (4167.239861027674 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07082867615360289 = (3801.907878392583 - 4071.1919802770826) / 3801.907878392583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareTo(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2311089427662958 > 1.185497018141746.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.963227112844246 (T) = (0 -1235.42896724949) / Math.Sqrt((58.00855052658225 / (19)) + (1437.8507476580194 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.0926696156589085 = (1130.6518910608618 - 1235.42896724949) / 1130.6518910608618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsIdentityBenchmark - Duration of single invocation 39.76 ns 42.09 ns 1.06 0.20 False
EqualsBenchmark - Duration of single invocation 84.86 ns 99.04 ns 1.17 0.19 False
MultiplyByMatrixBenchmark - Duration of single invocation 70.78 ns 76.65 ns 1.08 0.21 False
InvertBenchmark - Duration of single invocation 52.80 ns 59.14 ns 1.12 0.19 False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 67.19 ns 73.62 ns 1.10 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix3x2.IsIdentityBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.087073057287625 > 41.51278756976847.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.349136015198312 (T) = (0 -42.89531873103531) / Math.Sqrt((5.361010588697712 / (19)) + (1.4538008301667096 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07767605399825658 = (39.80353703869596 - 42.89531873103531) / 39.80353703869596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.EqualsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.03907624874742 > 90.1808980437297.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.33569079717891 (T) = (0 -101.8903555178933) / Math.Sqrt((13.766572693245609 / (19)) + (3.936566754034489 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.16278523486037247 = (87.62611741464735 - 101.8903555178933) / 87.62611741464735 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.65142460966072 > 74.81260214181067.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.391875067231139 (T) = (0 -76.94794769802438) / Math.Sqrt((4.069316206313667 / (19)) + (1.867846722517604 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07709264249272628 = (71.44041715849342 - 76.94794769802438) / 71.44041715849342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.InvertBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.139862789409904 > 55.873726297913045.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.48889453626604 (T) = (0 -60.21054570716405) / Math.Sqrt((0.8031140548845939 / (19)) + (3.8623100375768162 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12259062958809631 = (53.63535390390409 - 60.21054570716405) / 53.63535390390409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.61949223282168 > 71.01678818577413.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.322517568844944 (T) = (0 -73.35480910863349) / Math.Sqrt((1.2373812838354168 / (19)) + (1.4963155239712762 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08297671517656327 = (67.73442870992298 - 73.35480910863349) / 67.73442870992298 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EmptyEnumerable - Duration of single invocation 293.38 ns 314.57 ns 1.07 0.21 False
Singleton - Duration of single invocation 282.29 ns 306.39 ns 1.09 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.EmptyEnumerable

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 314.56645963059736 > 303.13815227890564.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.29371744228477 (T) = (0 -312.13338530047815) / Math.Sqrt((53.10342615079663 / (19)) + (71.48728998009507 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07394857961626801 = (290.6409033214666 - 312.13338530047815) / 290.6409033214666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.GetService.Singleton

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.39282357887333 > 294.6273891070225.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.674206519183366 (T) = (0 -302.4608804951773) / Math.Sqrt((110.1507636342736 / (19)) + (45.21885980450105 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06391704343754265 = (284.2899099707235 - 302.4608804951773) / 284.2899099707235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyByMatrixBenchmark - Duration of single invocation 233.44 ns 271.61 ns 1.16 0.08 False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 229.57 ns 265.14 ns 1.15 0.06 False
GetDeterminantBenchmark - Duration of single invocation 75.51 ns 90.12 ns 1.19 0.19 False
IsIdentityBenchmark - Duration of single invocation 57.03 ns 61.23 ns 1.07 0.08 False
EqualsBenchmark - Duration of single invocation 179.73 ns 219.18 ns 1.22 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 271.60990821476554 > 245.58430396723972.
IsChangePoint: Marked as a change because one of 9/16/2022 2:24:22 AM, 9/16/2022 6:13:21 PM, 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.913350447613563 (T) = (0 -270.51457504192143) / Math.Sqrt((4.641232766859231 / (19)) + (63.24087828088451 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.16259103120803464 = (232.68248918180018 - 270.51457504192143) / 232.68248918180018 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 265.14211738889486 > 243.76503471251095.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.70535001987682 (T) = (0 -266.66914648062846) / Math.Sqrt((8.518638346633255 / (19)) + (43.45523183610704 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.15078669528244326 = (231.72769338906767 - 266.66914648062846) / 231.72769338906767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.GetDeterminantBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.12453637319858 > 78.74759678824203.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.354660322632926 (T) = (0 -85.47598298293009) / Math.Sqrt((2.943358895931935 / (19)) + (8.039984582423406 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12569179503000485 = (75.93195878331134 - 85.47598298293009) / 75.93195878331134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.IsIdentityBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.2305247306967 > 60.01219731762071.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.726155090273709 (T) = (0 -62.71470390031953) / Math.Sqrt((1.5322100315225546 / (19)) + (2.2929605520376093 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07405848931608486 = (58.390399148796455 - 62.71470390031953) / 58.390399148796455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.EqualsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.17845372166752 > 185.88447684525948.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -25.70768768925828 (T) = (0 -220.6920140220229) / Math.Sqrt((35.82558376099006 / (19)) + (13.14081244924039 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.21864170730722807 = (181.09671833706977 - 220.6920140220229) / 181.09671833706977 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CtorDefaultSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 88.79 ns 98.25 ns 1.11 0.19 False
SortedList - Duration of single invocation 62.89 ns 67.68 ns 1.08 0.12 False
SortedDictionary - Duration of single invocation 152.86 ns 164.82 ns 1.08 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<String>.Dictionary

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.25404291417377 > 93.16337881506453.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.917389018670132 (T) = (0 -98.71976632403025) / Math.Sqrt((1.3044562088855767 / (19)) + (5.508434705688044 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10314330730268308 = (89.48952114427621 - 98.71976632403025) / 89.48952114427621 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.SortedList

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.67899655284135 > 65.79651482181063.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.560065980352328 (T) = (0 -68.69783040901154) / Math.Sqrt((13.32968267127546 / (19)) + (2.000412147260806 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07675128675084597 = (63.801020025999584 - 68.69783040901154) / 63.801020025999584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.SortedDictionary

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 164.81727669518926 > 159.53413297702218.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/12/2022 11:17:15 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.218573224260572 (T) = (0 -163.33245834943972) / Math.Sqrt((4.584230511601369 / (19)) + (24.747730606473265 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07858106002850226 = (151.43271507579 - 163.33245834943972) / 151.43271507579 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 100.27 μs 115.52 μs 1.15 0.04 False
Array - Duration of single invocation 2.22 μs 2.42 μs 1.09 0.04 False
ImmutableList - Duration of single invocation 102.48 μs 115.34 μs 1.13 0.04 False
IList - Duration of single invocation 17.62 μs 19.00 μs 1.08 0.14 False
List - Duration of single invocation 8.96 μs 10.54 μs 1.18 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.52015118827158 > 106.15302253415268.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.350782878388916 (T) = (0 -115381.29945334485) / Math.Sqrt((8780268.945335362 / (19)) + (3484329.0568273426 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13016073421481494 = (102092.82269349643 - 115381.29945334485) / 102092.82269349643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.416933600154679 > 2.3239565570186267.
IsChangePoint: Marked as a change because one of 10/27/2022 9:35:54 AM, 10/27/2022 9:29:16 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -3.108262269158012 (T) = (0 -2427.864717069305) / Math.Sqrt((51066.32395874191 / (19)) + (3867.010782698767 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07302551123736395 = (2262.6346639881863 - 2427.864717069305) / 2262.6346639881863 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;Int32&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.34345805759806 > 107.08227197950487.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.034092247614165 (T) = (0 -115842.8825603848) / Math.Sqrt((8069958.293636395 / (19)) + (6328719.243141962 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13458519502045202 = (102101.52844299772 - 115842.8825603848) / 102101.52844299772 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;Int32&gt;.IList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.998937798705242 > 18.03109866646851.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.295475218855604 (T) = (0 -19235.373198241876) / Math.Sqrt((127032.63594531883 / (19)) + (343194.2637322371 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.1337692988322539 = (16965.86176575225 - 19235.373198241876) / 16965.86176575225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.536101828872141 > 9.405816334531222.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.906419852434963 (T) = (0 -10545.734175066957) / Math.Sqrt((113206.34895998848 / (19)) + (105896.73170232316 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.16211829675610034 = (9074.578900017304 - 10545.734175066957) / 9074.578900017304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Divide - Duration of single invocation 59.03 ns 67.65 ns 1.15 0.20 False
Multiply - Duration of single invocation 17.03 ms 18.19 ms 1.07 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.65162916635046 > 62.13313158430726.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.141464809552708 (T) = (0 -65.14959698546723) / Math.Sqrt((3.3759043169809138 / (19)) + (2.407310464321026 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.078145881287695 = (60.42744132886276 - 65.14959698546723) / 60.42744132886276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 65536,65536 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.19224668717949 > 17.84831017232143.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.854733199765624 (T) = (0 -18254058.89358362) / Math.Sqrt((82772809336.56235 / (19)) + (236314438748.68802 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07211804139592026 = (17026165.2064137 - 18254058.89358362) / 17026165.2064137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetTypeCode - Duration of single invocation 24.80 ns 26.63 ns 1.07 0.25 False
FromBase64String - Duration of single invocation 824.82 ns 887.14 ns 1.08 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.GetTypeCode

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.633388512376182 > 25.97255584851225.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.881730545198526 (T) = (0 -26.975255197474844) / Math.Sqrt((0.33649293379776435 / (19)) + (0.6644076122379154 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07265290965171468 = (25.14816764561202 - 26.975255197474844) / 25.14816764561202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.FromBase64String

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 887.1387776945065 > 870.2969617625841.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 10/11/2022 10:45:23 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.06599087213383 (T) = (0 -897.446819111154) / Math.Sqrt((78.80964078332316 / (19)) + (223.07126282010688 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07984913067658642 = (831.0853744437919 - 897.446819111154) / 831.0853744437919 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchstone.BenchF.DMath

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 8.68 secs 9.63 secs 1.11 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.DMath*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.DMath.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.627727252533333 > 9.136721056324618.
IsChangePoint: Marked as a change because one of 7/31/2022 5:42:25 AM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.150934606742764 (T) = (0 -9755328870.414082) / Math.Sqrt((7495322252424466 / (19)) + (3.555100061420702E+17 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12023275642464257 = (8708305318.217426 - 9755328870.414082) / 8708305318.217426 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentBag - Duration of single invocation 869.12 ms 940.98 ms 1.08 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>.ConcurrentBag(Size: 2000000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 940.9769728 > 912.8579808600001.
IsChangePoint: Marked as a change because one of 8/4/2022 12:00:02 PM, 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.104971667483893 (T) = (0 -951027587.3365058) / Math.Sqrt((219182611567780.28 / (19)) + (545756934015105.7 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0831377523357406 = (878030135.3965875 - 951027587.3365058) / 878030135.3965875 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateGetPositionTenSegments - Duration of single invocation 889.85 ns 978.50 ns 1.10 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionTenSegments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 978.500095642626 > 936.1854469928558.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.049856372011796 (T) = (0 -964.4348556866297) / Math.Sqrt((508.12299892772836 / (19)) + (223.17676037025376 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06541182457177677 = (905.2225941590868 - 964.4348556866297) / 905.2225941590868 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddChainedConfigurationWithSplitting - Duration of single invocation 3.99 ms 4.38 ms 1.10 0.04 False
AddChainedConfigurationWithCommonPaths - Duration of single invocation 4.01 ms 4.37 ms 1.09 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Benchmark.GetChildKeysTests.AddChainedConfigurationWithSplitting

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.383982876041666 > 4.2265121813085935.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.792429064591808 (T) = (0 -4338006.164055346) / Math.Sqrt((2968400034.204327 / (19)) + (3002640799.632204 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0756348924936167 = (4032972.706936512 - 4338006.164055346) / 4032972.706936512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Benchmark.GetChildKeysTests.AddChainedConfigurationWithCommonPaths

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.36676565625 > 4.20287761890625.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.479823789219605 (T) = (0 -4354919.321719955) / Math.Sqrt((2665301515.635746 / (19)) + (2684372557.0056896 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08201174818074883 = (4024835.524237275 - 4354919.321719955) / 4024835.524237275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsBenchmark - Duration of single invocation 35.10 ns 38.26 ns 1.09 0.29 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.26063602935175 > 37.00804039581241.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.2556430623993 (T) = (0 -39.48442119707991) / Math.Sqrt((0.3613715919077985 / (19)) + (2.73938292969935 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.10981153936400152 = (35.57759114642762 - 39.48442119707991) / 35.57759114642762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetElement - Duration of single invocation 462.72 ns 510.86 ns 1.10 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElement.GetElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 510.8636539250009 > 497.6648582119192.
IsChangePoint: Marked as a change because one of 7/31/2022 3:07:22 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.8080370055430794 (T) = (0 -512.8066337540647) / Math.Sqrt((159.76469159131963 / (19)) + (205.02060352545664 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06427119708583337 = (481.83830884291694 - 512.8066337540647) / 481.83830884291694 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 26.41 μs 29.72 μs 1.13 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEachNonGeneric<Int32>.ArrayList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.719402953551548 > 27.78419048092549.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.800315732913134 (T) = (0 -29062.903134851575) / Math.Sqrt((2184310.5127837663 / (19)) + (331263.05231619894 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07650294032971425 = (26997.513937073047 - 29062.903134851575) / 26997.513937073047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IterateForNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 17.13 μs 19.46 μs 1.14 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForNonGeneric<Int32>.ArrayList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.45675874125874 > 17.743142924370133.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.744342167387057 (T) = (0 -19585.73283680696) / Math.Sqrt((320739.1490731852 / (19)) + (258964.26955665392 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13854950163882923 = (17202.355109387197 - 19585.73283680696) / 17202.355109387197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchstone.BenchF.NewtR

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 3.12 secs 3.48 secs 1.12 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.NewtR*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.NewtR.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.482080884928571 > 3.2768094306475.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.989211215550732 (T) = (0 -3477430367.0198064) / Math.Sqrt((641326715239819.9 / (19)) + (5221646627474551 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10878318099416574 = (3136258221.288896 - 3477430367.0198064) / 3136258221.288896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NoArguments - Duration of single invocation 224.25 ns 258.28 ns 1.15 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.Formatting*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.Formatting.NoArguments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 258.2805996855393 > 235.05162964938557.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.421272415933494 (T) = (0 -250.22659596274397) / Math.Sqrt((45.017928647378 / (19)) + (38.46274238699138 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10624157030876163 = (226.19525669506666 - 250.22659596274397) / 226.19525669506666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCount - Duration of single invocation 119.11 ns 140.36 ns 1.18 0.20 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.GetCount

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 140.35620360679277 > 125.01615881503228.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 8/1/2022 12:35:11 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.060603262131536 (T) = (0 -138.2031100933873) / Math.Sqrt((6.2271334281195445 / (19)) + (12.725781703341234 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14728806895882735 = (120.46068797595679 - 138.2031100933873) / 120.46068797595679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 1.86 μs 2.01 μs 1.08 0.06 False
SerializeToUtf8Bytes - Duration of single invocation 1.53 μs 1.70 μs 1.11 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0135852979274613 > 1.9620173247802128.
IsChangePoint: Marked as a change because one of 7/31/2022 3:34:07 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.64175581166043 (T) = (0 -2039.403822348025) / Math.Sqrt((809.1082922945819 / (19)) + (2510.5811548280276 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08333445117033776 = (1882.5246627621186 - 2039.403822348025) / 1882.5246627621186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7015521969324996 > 1.5898319980664082.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.503829740592657 (T) = (0 -1658.1673848830383) / Math.Sqrt((640.0313133361661 / (19)) + (1192.3297166740108 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.09627040927670832 = (1512.5532631835383 - 1658.1673848830383) / 1512.5532631835383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayNot - Duration of single invocation 732.83 ns 825.55 ns 1.13 0.06 False
BitArrayLengthCtor - Duration of single invocation 96.86 ns 103.01 ns 1.06 0.08 False
BitArrayLengthCtor - Duration of single invocation 102.07 ns 110.04 ns 1.08 0.07 False
BitArrayNot - Duration of single invocation 30.45 ns 33.44 ns 1.10 0.25 False
BitArrayAnd - Duration of single invocation 46.66 ns 50.61 ns 1.08 0.18 False
BitArrayLeftShift - Duration of single invocation 65.01 ns 69.43 ns 1.07 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 825.5492868507052 > 769.4486626410937.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.205055998281307 (T) = (0 -796.6342848806975) / Math.Sqrt((1.5674690049132807 / (19)) + (2216.9749142460037 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08757169894496948 = (732.4889804079085 - 796.6342848806975) / 732.4889804079085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 103.01320999108039 > 102.60694538609323.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.84856932640645 (T) = (0 -104.17582465245367) / Math.Sqrt((4.292325296683408 / (19)) + (2.7224914987745876 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05073370988077684 = (99.1457908629144 - 104.17582465245367) / 99.1457908629144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.04327838993751 > 108.8728189533869.
IsChangePoint: Marked as a change because one of 8/3/2022 1:54:17 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.802197418228351 (T) = (0 -109.23824297433364) / Math.Sqrt((2.8659993259552947 / (19)) + (3.1725429507000382 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.05928526562510213 = (103.12448074114418 - 109.23824297433364) / 103.12448074114418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.44312111785694 > 32.06136412481337.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.05754248474086 (T) = (0 -34.56261263651643) / Math.Sqrt((0.41394676884979204 / (19)) + (2.263268432607857 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12581433382586635 = (30.700100005888135 - 34.56261263651643) / 30.700100005888135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.612567981574756 > 49.07199257834012.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.032678626172864 (T) = (0 -50.974968406878816) / Math.Sqrt((1.508400591547505 / (19)) + (1.3242089357112736 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06731483116341981 = (47.76001130923466 - 50.974968406878816) / 47.76001130923466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.43480850800704 > 68.73447174451177.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.010103229093982 (T) = (0 -71.665459068923) / Math.Sqrt((1.4909179732176114 / (19)) + (11.038470747679131 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09461840922488732 = (65.47072337260451 - 71.665459068923) / 65.47072337260451 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 1.18 ms 1.31 ms 1.10 0.01 False
LinkedList - Duration of single invocation 153.59 μs 165.81 μs 1.08 0.03 False
SortedDictionary - Duration of single invocation 1.40 ms 1.54 ms 1.10 0.04 False
Stack - Duration of single invocation 27.90 μs 30.16 μs 1.08 0.06 False
Dictionary - Duration of single invocation 192.07 μs 211.49 μs 1.10 0.00 False
Queue - Duration of single invocation 45.82 μs 51.32 μs 1.12 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3055380704861108 > 1.2526413214543268.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.09514809580921 (T) = (0 -1313850.9005398436) / Math.Sqrt((236190359.99239546 / (19)) + (466387828.60471225 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09974676346040066 = (1194684.9440190713 - 1313850.9005398436) / 1194684.9440190713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;Int32&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 165.80993437499998 > 159.61504696231972.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 9:30:44 PM, 10/23/2022 8:43:59 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.752081626566888 (T) = (0 -166029.58401510696) / Math.Sqrt((4451665.539258845 / (19)) + (3237763.1698268494 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08401206360054665 = (153162.1183842176 - 166029.58401510696) / 153162.1183842176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;Int32&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5408531036931818 > 1.4841981485227276.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/14/2022 1:19:37 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.5047046275015 (T) = (0 -1544215.31473464) / Math.Sqrt((332266423.2452266 / (19)) + (534204360.57027006 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08730483246310818 = (1420222.984971452 - 1544215.31473464) / 1420222.984971452 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.163780415860735 > 29.311259128823508.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.057151918558638 (T) = (0 -30186.857143171852) / Math.Sqrt((172173.23192082648 / (19)) + (163998.56320197773 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07471063414086869 = (28088.358097715707 - 30186.857143171852) / 28088.358097715707 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;Int32&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 211.49243001126123 > 203.7518361682997.
IsChangePoint: Marked as a change because one of 7/31/2022 5:42:25 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -29.000830936001606 (T) = (0 -211070.0698383113) / Math.Sqrt((2797845.1851582695 / (19)) + (5396624.469083763 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08709328968047178 = (194160.03377258533 - 211070.0698383113) / 194160.03377258533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.319530296840945 > 48.135611315596776.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.636738102857228 (T) = (0 -50644.290712562986) / Math.Sqrt((2925179.6425949107 / (19)) + (741378.3311833559 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08793514971989812 = (46550.835980988355 - 50644.290712562986) / 46550.835980988355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 121.02 μs 128.64 μs 1.06 0.06 False
Stack - Duration of single invocation 15.24 μs 16.42 μs 1.08 0.13 False
Queue - Duration of single invocation 24.71 μs 27.39 μs 1.11 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.IDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.63662745535714 > 126.78673416502302.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.841516079735929 (T) = (0 -128711.61791039696) / Math.Sqrt((10016161.605462275 / (19)) + (2898110.284194628 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05084178927192511 = (122484.29708869373 - 128711.61791039696) / 122484.29708869373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.AddGivenSize&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.417804017857147 > 16.022207877860538.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.65662870622103 (T) = (0 -16562.072404706214) / Math.Sqrt((146919.72890473576 / (19)) + (101877.65328380464 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06629643005763933 = (15532.334103201434 - 16562.072404706214) / 15532.334103201434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.AddGivenSize&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.393665964673914 > 25.94219849083045.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.30443218406775 (T) = (0 -27591.250425721817) / Math.Sqrt((106293.33372081087 / (19)) + (316954.09516986576 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11163095027439435 = (24820.512975921738 - 27591.250425721817) / 24820.512975921738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in BenchmarksGame.Fasta_2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 2.37 ms 2.52 ms 1.06 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.Fasta_2*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.Fasta_2.RunBench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5155200114583343 > 2.4903739946874994.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.950948358894436 (T) = (0 -2526295.9125039456) / Math.Sqrt((504592685.3457417 / (19)) + (444484201.4223883 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.059995629670148036 = (2383307.8569296408 - 2526295.9125039456) / 2383307.8569296408 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 312.48 ns 329.78 ns 1.06 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "ascii")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 329.77582134185405 > 328.362969922246.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.587920751911307 (T) = (0 -334.5668301135093) / Math.Sqrt((61.43665413652572 / (19)) + (29.045203445244336 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05601872964800325 = (316.81903049676833 - 334.5668301135093) / 316.81903049676833 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConvertTryFromBase64Chars - Duration of single invocation 13.37 μs 14.28 μs 1.07 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64Tests.ConvertTryFromBase64Chars(NumberOfBytes: 1000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.278727470645794 > 14.102209281176277.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/13/2022 5:34:08 PM, 10/14/2022 6:08:00 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.214277032510725 (T) = (0 -14453.376462184548) / Math.Sqrt((52673.81056792057 / (19)) + (518631.9467683003 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.06808472701205213 = (13532.050498107588 - 14453.376462184548) / 13532.050498107588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NoArguments_DefineMessage_FilteredByLevel - Duration of single invocation 64.73 ns 69.89 ns 1.08 0.22 False
NoArguments_FilteredByLevel - Duration of single invocation 124.21 ns 135.85 ns 1.09 0.18 False
NoArguments_DefineMessage - Duration of single invocation 204.40 ns 218.49 ns 1.07 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage_FilteredByLevel

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.89055974297592 > 68.85253852657877.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.810287090123696 (T) = (0 -70.22281290273064) / Math.Sqrt((4.309352037218208 / (19)) + (2.803193933547246 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05870631528345761 = (66.32888827524299 - 70.22281290273064) / 66.32888827524299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.84674610108223 > 131.60351542213024.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.429786954122926 (T) = (0 -137.6989421496729) / Math.Sqrt((14.356000849875866 / (19)) + (11.336202533186052 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07964613129809106 = (127.54081004681905 - 137.6989421496729) / 127.54081004681905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 218.4873837839605 > 215.2818782231647.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.574624774590347 (T) = (0 -217.96170313530794) / Math.Sqrt((5.769345841591206 / (19)) + (18.47010830034673 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.06082427548442575 = (205.46447528812033 - 217.96170313530794) / 205.46447528812033 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPinnableReference - Duration of single invocation 18.57 ns 22.40 ns 1.21 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.GetPinnableReference(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.39608501255446 > 19.417662366222014.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.612917136414897 (T) = (0 -22.89841630199742) / Math.Sqrt((0.4034684606759599 / (19)) + (1.7271960754182443 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.22513756103152335 = (18.690485893451626 - 22.89841630199742) / 18.690485893451626 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 73.21 μs 78.31 μs 1.07 0.10 False
ImmutableQueue - Duration of single invocation 34.57 μs 38.60 μs 1.12 0.09 False
SortedDictionary - Duration of single invocation 84.11 μs 94.10 μs 1.12 0.13 False
HashSet - Duration of single invocation 16.68 μs 18.21 μs 1.09 0.08 False
LinkedList - Duration of single invocation 17.84 μs 19.68 μs 1.10 0.07 False
ImmutableSortedSet - Duration of single invocation 142.49 μs 160.52 μs 1.13 0.06 False
ImmutableHashSet - Duration of single invocation 275.57 μs 292.09 μs 1.06 0.05 False
Queue - Duration of single invocation 25.48 μs 27.07 μs 1.06 0.06 False
ImmutableArray - Duration of single invocation 10.79 μs 11.50 μs 1.07 0.13 False
ImmutableList - Duration of single invocation 150.87 μs 169.06 μs 1.12 0.06 False
ImmutableSortedDictionary - Duration of single invocation 145.00 μs 158.11 μs 1.09 0.05 False
IEnumerable - Duration of single invocation 20.98 μs 22.30 μs 1.06 0.08 False
ImmutableDictionary - Duration of single invocation 264.36 μs 285.70 μs 1.08 0.02 False
ConcurrentBag - Duration of single invocation 28.42 μs 31.23 μs 1.10 0.04 False
ConcurrentStack - Duration of single invocation 23.54 μs 25.87 μs 1.10 0.09 False
ImmutableStack - Duration of single invocation 34.43 μs 38.23 μs 1.11 0.10 False
ConcurrentQueue - Duration of single invocation 37.81 μs 43.58 μs 1.15 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.31358929086538 > 78.09341664630004.
IsChangePoint: Marked as a change because one of 10/2/2022 5:38:34 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.967690283597621 (T) = (0 -79034.07313232248) / Math.Sqrt((1321553.256103738 / (19)) + (1177057.608120903 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06274178442532354 = (74368.08667032892 - 79034.07313232248) / 74368.08667032892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableQueue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.60492909185083 > 36.53174357248913.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.884836522198414 (T) = (0 -39142.457122754) / Math.Sqrt((2139347.8867475013 / (19)) + (963129.7828203281 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11934022381318682 = (34969.22230616338 - 39142.457122754) / 34969.22230616338 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 94.10345520231215 > 87.7133479255579.
IsChangePoint: Marked as a change because one of 10/3/2022 5:29:39 AM, 11/15/2022 2:18:08 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.535157836965368 (T) = (0 -90831.4664136448) / Math.Sqrt((2003142.047168851 / (17)) + (1707288.6713187196 / (30))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (30) - 2, .025) and -0.08778371266350501 = (83501.40322586591 - 90831.4664136448) / 83501.40322586591 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.HashSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.211930210767132 > 17.48998445497282.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.4351745743375 (T) = (0 -18338.116827959562) / Math.Sqrt((20117.16274083285 / (19)) + (170763.3224384934 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09295555289856744 = (16778.465308425442 - 18338.116827959562) / 16778.465308425442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.677401351710476 > 18.740215748013345.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.150486859836123 (T) = (0 -19443.892723332832) / Math.Sqrt((86395.50819617411 / (19)) + (199911.51831633897 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07888258709069003 = (18022.25094369643 - 19443.892723332832) / 18022.25094369643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 160.52368869661262 > 152.57733616956511.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.585890228498204 (T) = (0 -158621.61371058293) / Math.Sqrt((6831960.3805490285 / (19)) + (3052478.530632288 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09232775158319921 = (145214.30356473115 - 158621.61371058293) / 145214.30356473115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableHashSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 292.08761329177895 > 290.03583440926536.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.342902055998362 (T) = (0 -296793.8511865254) / Math.Sqrt((15120307.448359154 / (19)) + (14632648.720439283 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07638430722563878 = (275732.2354053138 - 296793.8511865254) / 275732.2354053138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.072988571428574 > 26.78722137064485.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/15/2022 1:58:20 AM, 10/16/2022 12:49:12 AM, 11/15/2022 9:22:46 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.873559522556638 (T) = (0 -27408.63435796704) / Math.Sqrt((300313.4512463257 / (16)) + (340096.5029471902 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.059133691149758895 = (25878.35188984799 - 27408.63435796704) / 25878.35188984799 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.502237486537044 > 11.33726247660016.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.633740976585564 (T) = (0 -11591.690053124374) / Math.Sqrt((63834.88030438614 / (19)) + (88162.37547681695 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0639295745824283 = (10895.166682131086 - 11591.690053124374) / 10895.166682131086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 169.0638738253546 > 158.3388337094351.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.522190072768417 (T) = (0 -167586.96150203634) / Math.Sqrt((8330061.665495709 / (19)) + (9473855.457972694 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09515964105315693 = (153025.14375061964 - 167586.96150203634) / 153025.14375061964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 158.10598556250002 > 155.1890962024366.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.170213341864642 (T) = (0 -158092.04068678673) / Math.Sqrt((11056529.701735212 / (19)) + (1277930.5733314569 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.0537750401321615 = (150024.4688533895 - 158092.04068678673) / 150024.4688533895 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.IEnumerable(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.29550710082535 > 22.143362528735572.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.42765203933308 (T) = (0 -22518.521744932936) / Math.Sqrt((249949.5201137556 / (19)) + (124478.84714370355 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07254763141241219 = (20995.358234374016 - 22518.521744932936) / 20995.358234374016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 285.6966648919753 > 277.39246986562136.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.12046117836467 (T) = (0 -282335.2550290803) / Math.Sqrt((10966984.47351389 / (19)) + (14461211.601495817 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06350335306478172 = (265476.60072293377 - 282335.2550290803) / 265476.60072293377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ConcurrentBag(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.2257773125 > 30.348913253914915.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.640616142519258 (T) = (0 -31741.30910085115) / Math.Sqrt((240790.77876458789 / (19)) + (981294.956137444 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10360231470751674 = (28761.546326824642 - 31741.30910085115) / 28761.546326824642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ConcurrentStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.874514055601544 > 25.597562223606737.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.106534531917148 (T) = (0 -25969.07779156775) / Math.Sqrt((330353.01304543164 / (19)) + (268032.0253501943 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08289041878092408 = (23981.26102251669 - 25969.07779156775) / 23981.26102251669 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.2298111144822 > 35.85530102486721.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.78263068007236 (T) = (0 -38258.77211509757) / Math.Sqrt((352587.77780859935 / (18)) + (278071.2736199387 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.11323453374732541 = (34367.216390882546 - 38258.77211509757) / 34367.216390882546 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ConcurrentQueue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.58031424689442 > 40.33977591783217.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 2:18:08 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.489530049491663 (T) = (0 -41347.1477008647) / Math.Sqrt((599358.1731586278 / (17)) + (481708.0562121533 / (30))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (30) - 2, .025) and -0.07345100353017839 = (38517.96455067759 - 41347.1477008647) / 38517.96455067759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LerpBenchmark - Duration of single invocation 139.62 ns 148.20 ns 1.06 0.30 False
NegateBenchmark - Duration of single invocation 45.08 ns 50.53 ns 1.12 0.26 False
EqualsBenchmark - Duration of single invocation 103.62 ns 113.06 ns 1.09 0.19 False
SubtractionOperatorBenchmark - Duration of single invocation 86.87 ns 92.80 ns 1.07 0.28 False
SlerpBenchmark - Duration of single invocation 138.54 ns 149.11 ns 1.08 0.24 False
LengthBenchmark - Duration of single invocation 61.29 ns 64.86 ns 1.06 0.12 False
MultiplyByQuaternionOperatorBenchmark - Duration of single invocation 107.98 ns 118.48 ns 1.10 0.23 False
MultiplyByScalarBenchmark - Duration of single invocation 48.65 ns 51.31 ns 1.05 0.21 False
LengthSquaredBenchmark - Duration of single invocation 49.47 ns 55.36 ns 1.12 0.22 False
ConcatenateBenchmark - Duration of single invocation 108.55 ns 116.13 ns 1.07 0.21 False
InverseBenchmark - Duration of single invocation 59.38 ns 66.53 ns 1.12 0.22 False
DivisionOperatorBenchmark - Duration of single invocation 132.50 ns 144.62 ns 1.09 0.21 False
AddOperatorBenchmark - Duration of single invocation 85.09 ns 91.86 ns 1.08 0.21 False
NormalizeBenchmark - Duration of single invocation 62.13 ns 67.75 ns 1.09 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Quaternion.LerpBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.19768766718784 > 145.37554183964565.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/17/2022 12:47:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.180199731728939 (T) = (0 -151.2485563559815) / Math.Sqrt((6.9288142083048925 / (19)) + (15.923143433263975 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09192825446142024 = (138.5151045757882 - 151.2485563559815) / 138.5151045757882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.53438994341842 > 47.348005047543886.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.565842249779844 (T) = (0 -49.683463003246914) / Math.Sqrt((6.245455947253284 / (19)) + (3.3191770224751105 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.08141571506779512 = (45.94298225093964 - 49.683463003246914) / 45.94298225093964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.EqualsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.05605144633871 > 107.67494814429475.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 10/17/2022 12:47:34 PM, 11/15/2022 9:22:46 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.805768522023937 (T) = (0 -115.48180831133364) / Math.Sqrt((5.665666175307981 / (16)) + (21.098869875279096 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.138441991297403 = (101.43846519551433 - 115.48180831133364) / 101.43846519551433 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.SubtractionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.79968448484237 > 91.05575411704865.
IsChangePoint: Marked as a change because one of 10/20/2022 10:19:58 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.044336157616907 (T) = (0 -93.07328657151086) / Math.Sqrt((4.303934779972759 / (19)) + (10.052823898500783 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08033263050840818 = (86.15243485491155 - 93.07328657151086) / 86.15243485491155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.SlerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 149.11326435970113 > 144.76208776630475.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.251207614569719 (T) = (0 -149.9853893535587) / Math.Sqrt((6.837188321825181 / (19)) + (16.459003267630376 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0863860265128954 = (138.0590192557842 - 149.9853893535587) / 138.0590192557842 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.LengthBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.85963110248973 > 64.54714693667874.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.159119105571884 (T) = (0 -65.8466683577245) / Math.Sqrt((2.347226856481282 / (19)) + (1.1165473244135105 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05957986062461847 = (62.14412976753647 - 65.8466683577245) / 62.14412976753647 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.47987412236424 > 114.16870876452379.
IsChangePoint: Marked as a change because one of 10/20/2022 10:19:58 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.409336523962972 (T) = (0 -117.19999811180625) / Math.Sqrt((7.047141747406749 / (19)) + (16.133897325711658 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0750709870475738 = (109.01605524084331 - 117.19999811180625) / 109.01605524084331 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.30714355926887 > 48.67324545973777.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 9:22:46 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.923431429629332 (T) = (0 -51.81772894827035) / Math.Sqrt((1.8352559927743468 / (16)) + (2.6251541275408767 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.10391112146916621 = (46.940127642982254 - 51.81772894827035) / 46.940127642982254 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.LengthSquaredBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.357550598378666 > 52.25028456775973.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.073530478419443 (T) = (0 -54.40625823104763) / Math.Sqrt((2.298706357875313 / (19)) + (4.28471731475016 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07302662050797468 = (50.70354937260691 - 54.40625823104763) / 50.70354937260691 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.ConcatenateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 116.13267093521033 > 114.39187570008903.
IsChangePoint: Marked as a change because one of 10/19/2022 12:02:11 AM, 11/15/2022 2:18:08 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.052937786257493 (T) = (0 -115.99831580184095) / Math.Sqrt((5.401929131465983 / (17)) + (5.835128898128724 / (30))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (30) - 2, .025) and -0.06613003641304123 = (108.8031589393292 - 115.99831580184095) / 108.8031589393292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.InverseBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.5317095410597 > 63.60239421221573.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 10/17/2022 12:47:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.966494234369002 (T) = (0 -66.32815018621096) / Math.Sqrt((0.7573125568925051 / (19)) + (4.041670964946611 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09158669456622 = (60.7630621703105 - 66.32815018621096) / 60.7630621703105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.DivisionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.61656346055142 > 138.8612857750822.
IsChangePoint: Marked as a change because one of 10/16/2022 1:15:28 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.907013798706522 (T) = (0 -142.32842047600133) / Math.Sqrt((12.601225843599444 / (18)) + (9.819236170312932 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.09320754374651873 = (130.19341230323866 - 142.32842047600133) / 130.19341230323866 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.85688818895811 > 90.11966107734425.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.13769734723654 (T) = (0 -92.71114400269086) / Math.Sqrt((7.415616959728403 / (18)) + (7.255781742262536 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.08719676752942389 = (85.27540439010892 - 92.71114400269086) / 85.27540439010892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.NormalizeBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.75335249724345 > 65.29674938897539.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.569529107640905 (T) = (0 -68.72159256859565) / Math.Sqrt((1.2096251593683054 / (19)) + (3.4471031994454244 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10086643313478097 = (62.42500497803983 - 68.72159256859565) / 62.42500497803983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
K_Max_Elements - Duration of single invocation 147.21 μs 165.83 μs 1.13 0.06 False
HeapSort - Duration of single invocation 6.55 μs 7.36 μs 1.12 0.04 False
Enumerate - Duration of single invocation 29.29 μs 30.83 μs 1.05 0.12 False
HeapSort - Duration of single invocation 2.18 ms 2.40 ms 1.10 0.04 False
Enumerate - Duration of single invocation 383.14 ns 441.29 ns 1.15 0.12 False
Enumerate - Duration of single invocation 3.03 μs 3.26 μs 1.07 0.11 False
HeapSort - Duration of single invocation 139.08 μs 151.76 μs 1.09 0.04 False
Dequeue_And_Enqueue - Duration of single invocation 5.00 ms 5.57 ms 1.11 0.02 False
Dequeue_And_Enqueue - Duration of single invocation 358.49 μs 387.67 μs 1.08 0.04 False
Dequeue_And_Enqueue - Duration of single invocation 16.76 μs 18.40 μs 1.10 0.03 False
K_Max_Elements - Duration of single invocation 4.34 μs 4.69 μs 1.08 0.03 False
K_Max_Elements - Duration of single invocation 21.32 μs 23.10 μs 1.08 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 1000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 165.82727030075188 > 154.7456293047506.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.35066293127836 (T) = (0 -163056.46998595376) / Math.Sqrt((5313687.564361635 / (19)) + (9828488.488444172 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11605541574327446 = (146100.6932862387 - 163056.46998595376) / 146100.6932862387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.HeapSort(Size: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.3612794850908365 > 6.8903303464027.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.46057810377586 (T) = (0 -7308.53083198958) / Math.Sqrt((6324.02781595817 / (19)) + (13069.946869480596 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10453452728207854 = (6616.842345321371 - 7308.53083198958) / 6616.842345321371 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.Enumerate(Size: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.83278568948413 > 30.71515036315854.
IsChangePoint: Marked as a change because one of 10/21/2022 3:39:11 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.37932892865687 (T) = (0 -31171.93363512397) / Math.Sqrt((54279.690943776186 / (18)) + (417900.63555079 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.060061961535033355 = (29405.765668626707 - 31171.93363512397) / 29405.765668626707 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.HeapSort(Size: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.396743551488095 > 2.2841310005859374.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.93821539756735 (T) = (0 -2429160.1842751317) / Math.Sqrt((859341928.5379041 / (19)) + (1416187487.6160462 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1067541035860878 = (2194850.8493478396 - 2429160.1842751317) / 2194850.8493478396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.Enumerate(Size: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 441.28725344737296 > 402.74009437479674.
IsChangePoint: Marked as a change because one of 10/12/2022 1:08:53 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.640275276424229 (T) = (0 -419.39943123739965) / Math.Sqrt((39.533236585701985 / (19)) + (813.0531392246178 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08110514203676317 = (387.9358398455734 - 419.39943123739965) / 387.9358398455734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.Enumerate(Size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.261028092298033 > 3.1845707493426056.
IsChangePoint: Marked as a change because one of 10/3/2022 2:18:04 AM, 10/3/2022 11:00:05 AM, 10/25/2022 2:56:59 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.26637493017617 (T) = (0 -3250.781350024322) / Math.Sqrt((2152.8103832172105 / (19)) + (17091.301460922754 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06398021768262038 = (3055.3024351379554 - 3250.781350024322) / 3055.3024351379554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.HeapSort(Size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 151.76156383495146 > 146.81384895676493.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.324925663235913 (T) = (0 -154561.47027634922) / Math.Sqrt((2570150.303690492 / (19)) + (10743012.220136669 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1046436064787692 = (139919.76178546762 - 154561.47027634922) / 139919.76178546762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.Dequeue_And_Enqueue(Size: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.571579772321428 > 5.292914893269232.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.028747789348778 (T) = (0 -5611856.166497144) / Math.Sqrt((6717251794.860353 / (19)) + (8745635381.874496 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1014395005257258 = (5095019.893347352 - 5611856.166497144) / 5095019.893347352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.Dequeue_And_Enqueue(Size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 387.67145364010986 > 374.7640385374273.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.38580796075628 (T) = (0 -390273.1146407359) / Math.Sqrt((19619757.826802127 / (19)) + (39775557.10852127 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09388421978333553 = (356777.3513709128 - 390273.1146407359) / 356777.3513709128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.Dequeue_And_Enqueue(Size: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.39848330896686 > 17.591943580333457.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.529553265773703 (T) = (0 -18564.372249356402) / Math.Sqrt((43056.342221715946 / (19)) + (95442.08785052485 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0957262196214889 = (16942.52808495295 - 18564.372249356402) / 16942.52808495295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.K_Max_Elements(Size: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.685765523423945 > 4.549176897245301.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.93124135212812 (T) = (0 -4742.368533883192) / Math.Sqrt((2776.82511467789 / (19)) + (5281.634772571474 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0924363616291813 = (4341.0936329607 - 4742.368533883192) / 4341.0936329607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.K_Max_Elements(Size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.098319464024648 > 21.75865096942782.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.14091085347308 (T) = (0 -23141.888693903322) / Math.Sqrt((85706.75191049022 / (19)) + (105893.03206123502 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10496735619661973 = (20943.50440682649 - 23141.888693903322) / 20943.50440682649 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseHex - Duration of single invocation 144.28 ns 179.33 ns 1.24 0.30 False
TryParseSpan - Duration of single invocation 99.75 ns 121.32 ns 1.22 0.23 False
ParseHex - Duration of single invocation 209.41 ns 264.83 ns 1.26 0.15 False
Parse - Duration of single invocation 173.60 ns 192.43 ns 1.11 0.20 False
Parse - Duration of single invocation 121.00 ns 132.10 ns 1.09 0.22 False
ParseSpan - Duration of single invocation 137.16 ns 155.86 ns 1.14 0.26 False
ParseHex - Duration of single invocation 301.16 ns 333.89 ns 1.11 0.13 False
TryParse - Duration of single invocation 102.22 ns 112.15 ns 1.10 0.26 False
ToString - Duration of single invocation 18.28 ns 22.64 ns 1.24 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ParseHex(value: "4")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 179.3261365465117 > 151.75405113356132.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.514462327298325 (T) = (0 -168.85751852837893) / Math.Sqrt((30.217073366796075 / (19)) + (44.55760397889068 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13847719324071406 = (148.3187537975357 - 168.85751852837893) / 148.3187537975357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParseSpan(value: "4")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.31582819516898 > 106.27860628019852.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.024475814615874 (T) = (0 -112.17067974709619) / Math.Sqrt((12.918353895587023 / (22)) + (10.309458646335338 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.07703251931520455 = (104.14790429764943 - 112.17067974709619) / 104.14790429764943 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ParseHex(value: "3039")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 264.82609231233266 > 223.5657583242326.
IsChangePoint: Marked as a change because one of 10/18/2022 9:26:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.084651540673267 (T) = (0 -241.47329215658687) / Math.Sqrt((22.359009065473344 / (19)) + (51.64324690667146 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.13260687602404436 = (213.2013298420595 - 241.47329215658687) / 213.2013298420595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 192.43118385714095 > 182.13923405298777.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.547791931528439 (T) = (0 -186.64758380343346) / Math.Sqrt((10.23889377585817 / (19)) + (21.26775966568551 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06189795587555219 = (175.76790949704719 - 186.64758380343346) / 175.76790949704719 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.Parse(value: "4")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.10099312080644 > 126.45802707780602.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.416400593708339 (T) = (0 -132.70821838560585) / Math.Sqrt((7.123342387193574 / (19)) + (31.990044250753936 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.09723223488337329 = (120.94815861813578 - 132.70821838560585) / 120.94815861813578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ParseSpan(value: "4")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.8647196619353 > 144.3457264333979.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.178701933506376 (T) = (0 -153.6327290387008) / Math.Sqrt((13.31720988231227 / (18)) + (17.122829962189936 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.09161114979427658 = (140.73942819991734 - 153.6327290387008) / 140.73942819991734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ParseHex(value: "80000000")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 333.8850672364287 > 315.9859595243422.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.03980473356251 (T) = (0 -337.8506873669676) / Math.Sqrt((37.174731377237784 / (19)) + (45.44916282775072 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10549258672805731 = (305.6109931654171 - 337.8506873669676) / 305.6109931654171 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryParse(value: "4")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 112.14762346585837 > 109.949601107698.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.781216319697181 (T) = (0 -114.65048255136536) / Math.Sqrt((6.925466610218314 / (18)) + (13.142311756707297 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.09418620944234435 = (104.78150936466048 - 114.65048255136536) / 104.78150936466048 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.636903987820492 > 19.18301458164785.
IsChangePoint: Marked as a change because one of 10/14/2022 11:50:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -33.87957133593926 (T) = (0 -22.810337495371975) / Math.Sqrt((0.20185370949104073 / (19)) + (0.16790885317559795 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.23683212900427317 = (18.442549284142316 - 22.810337495371975) / 18.442549284142316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 32.86 ms 35.33 ms 1.08 0.04 False
LinkedList - Duration of single invocation 24.35 ms 26.47 ms 1.09 0.05 False
ImmutableSortedSet - Duration of single invocation 584.28 μs 643.61 μs 1.10 0.04 False
SortedSet - Duration of single invocation 496.27 μs 547.51 μs 1.10 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.32889507142857 > 34.40255449000001.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.522361236159767 (T) = (0 -36149397.3962476) / Math.Sqrt((58173340227.58354 / (19)) + (1067164969419.7675 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10222496941725612 = (32796750.57203586 - 36149397.3962476) / 32796750.57203586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.473085407407403 > 25.592895418125.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.529267451849957 (T) = (0 -26877863.275787625) / Math.Sqrt((202693037243.90662 / (19)) + (286002866980.4389 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.10508855831248322 = (24321908.930838317 - 26877863.275787625) / 24321908.930838317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 643.6123453124999 > 611.402252349537.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.033102502523878 (T) = (0 -643609.0715456357) / Math.Sqrt((79212657.368969 / (19)) + (78008895.63515933 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10421658518154488 = (582864.8837400133 - 643609.0715456357) / 582864.8837400133 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.SortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 547.5136607142857 > 524.0194508098442.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.112891062786087 (T) = (0 -539111.4630685395) / Math.Sqrt((65842933.216425605 / (19)) + (153470400.99623355 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09152260725521104 = (493907.7390473955 - 539111.4630685395) / 493907.7390473955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinkedList - Duration of single invocation 12.31 ms 13.31 ms 1.08 0.07 False
SortedSet - Duration of single invocation 437.27 μs 487.66 μs 1.12 0.07 False
ImmutableList - Duration of single invocation 16.53 ms 17.89 ms 1.08 0.05 False
ImmutableSortedSet - Duration of single invocation 509.97 μs 555.57 μs 1.09 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.LinkedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.308917072916666 > 12.834820306601562.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.48463692035562 (T) = (0 -13387949.281924954) / Math.Sqrt((23030552420.947716 / (19)) + (32735017146.326363 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09798950760723153 = (12193148.649571648 - 13387949.281924954) / 12193148.649571648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.SortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 487.6633295454546 > 457.16415334635417.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.1195445818972 (T) = (0 -481821.98765567184) / Math.Sqrt((19160181.134251844 / (19)) + (92523599.19282731 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10538667658067111 = (435885.46692647645 - 481821.98765567184) / 435885.46692647645 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.887520579487184 > 17.20822855883333.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.228574171947802 (T) = (0 -17845223.933194786) / Math.Sqrt((18435908176.360188 / (19)) + (99861201243.30011 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09158426100584868 = (16348004.062234433 - 17845223.933194786) / 16348004.062234433 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 555.5669862882652 > 533.7548890150047.
IsChangePoint: Marked as a change because one of 8/2/2022 2:18:01 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -28.25086449164057 (T) = (0 -564665.6991660536) / Math.Sqrt((23667421.251128096 / (19)) + (75854873.00721282 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11048770587910113 = (508484.42191356304 - 564665.6991660536) / 508484.42191356304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 525.68 μs 569.10 μs 1.08 0.05 False
IDictionary - Duration of single invocation 100.64 μs 116.76 μs 1.16 0.05 False
ConcurrentDictionary - Duration of single invocation 117.16 μs 125.70 μs 1.07 0.08 False
Dictionary - Duration of single invocation 96.44 μs 108.72 μs 1.13 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 569.0980983024691 > 556.2436915490302.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.4069560126158 (T) = (0 -580514.0815953208) / Math.Sqrt((47089085.1834475 / (19)) + (182888666.41613945 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09890926592996184 = (528263.8881964978 - 580514.0815953208) / 528263.8881964978 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;.IDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 116.76073122970779 > 106.60085268927331.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.11443176733999 (T) = (0 -112710.69330201717) / Math.Sqrt((6788600.302945836 / (19)) + (6111251.0354480455 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12166195666534145 = (100485.43826617941 - 112710.69330201717) / 100485.43826617941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;.ConcurrentDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.70031296296297 > 122.87123866828716.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.51742995877545 (T) = (0 -126437.71343886484) / Math.Sqrt((9565833.270423982 / (18)) + (5286572.661521762 / (28))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (28) - 2, .025) and -0.09172571407966756 = (115814.54188376677 - 126437.71343886484) / 115814.54188376677 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.71976336185514 > 101.36748946029076.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.18491694564084 (T) = (0 -109662.25762049554) / Math.Sqrt((2872586.4358492186 / (19)) + (5135492.349304953 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1325295131900361 = (96829.49216184748 - 109662.25762049554) / 96829.49216184748 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateEntry - Duration of single invocation 125.83 ns 148.78 ns 1.18 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.CreateEntry

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.7802671903662 > 132.37435281661837.
IsChangePoint: Marked as a change because one of 9/17/2022 6:06:25 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.334322360730068 (T) = (0 -135.47929378192998) / Math.Sqrt((7.64732454406551 / (19)) + (13.781401565752033 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.061782834260738946 = (127.59604827879589 - 135.47929378192998) / 127.59604827879589 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Object

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetType_ - Duration of single invocation 3.33 ns 4.57 ns 1.37 0.66 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Object*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Object.GetType_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.57054550741473 > 3.5206254109571598.
IsChangePoint: Marked as a change because one of 10/12/2022 1:44:25 PM, 10/13/2022 5:34:08 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.543278224153053 (T) = (0 -4.733083046853631) / Math.Sqrt((0.4962723101808923 / (19)) + (0.1926009574843004 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.335342981955945 = (3.544469930804476 - 4.733083046853631) / 3.544469930804476 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 99.97 μs 107.43 μs 1.07 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.42864071839082 > 106.05884705942998.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.57332215728609 (T) = (0 -107571.67289668848) / Math.Sqrt((1339322.481920435 / (19)) + (1127652.837701563 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06797794807776322 = (100724.61991402075 - 107571.67289668848) / 100724.61991402075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 520.45 μs 572.68 μs 1.10 0.04 False
IDictionary - Duration of single invocation 101.23 μs 113.21 μs 1.12 0.06 False
SortedDictionary - Duration of single invocation 559.97 μs 642.36 μs 1.15 0.11 False
ConcurrentDictionary - Duration of single invocation 90.25 μs 101.44 μs 1.12 0.06 False
Dictionary - Duration of single invocation 98.50 μs 110.48 μs 1.12 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 572.6810178952992 > 558.4309787740384.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.011270797410466 (T) = (0 -582838.6068714054) / Math.Sqrt((67616188.91377486 / (19)) + (216705259.84870678 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0947762534960926 = (532381.4843536754 - 582838.6068714054) / 532381.4843536754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;.IDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.20979898897059 > 106.38036147746564.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/22/2022 11:45:45 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.264210346462537 (T) = (0 -114426.67675500175) / Math.Sqrt((3584337.9084806456 / (19)) + (4675747.324570542 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13797458690468042 = (100552.92804582324 - 114426.67675500175) / 100552.92804582324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 642.3587604166668 > 590.9019252455356.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/14/2022 1:19:37 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.863848347996154 (T) = (0 -623277.5205085396) / Math.Sqrt((75457581.70908827 / (19)) + (202901262.09409678 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10618890160825795 = (563445.8270213825 - 623277.5205085396) / 563445.8270213825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;.ConcurrentDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.43500031682026 > 94.47931934355807.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.017572060380036 (T) = (0 -102516.16616760369) / Math.Sqrt((4448050.0953499945 / (19)) + (3355446.1926146317 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13143694376876258 = (90607.0521492141 - 102516.16616760369) / 90607.0521492141 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.47679220273562 > 103.30304000351872.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -27.061783721358882 (T) = (0 -111019.66933076186) / Math.Sqrt((1388565.9868813483 / (19)) + (4481267.853877288 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13339331881018748 = (97953.3472522213 - 111019.66933076186) / 97953.3472522213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchstone.BenchI.Permutate

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 4.41 secs 4.88 secs 1.10 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Permutate*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Permutate.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.8784487118 > 4.6629794997153855.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.958828283644882 (T) = (0 -4906874264.8158045) / Math.Sqrt((20740714097698256 / (19)) + (10468380113979074 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09348996949644399 = (4487351874.910602 - 4906874264.8158045) / 4487351874.910602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in GuardedDevirtualization.TwoClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 17.30 ns 18.98 ns 1.10 0.05 False
Call - Duration of single invocation 17.64 ns 18.91 ns 1.07 0.07 False
Call - Duration of single invocation 17.34 ns 18.89 ns 1.09 0.07 False
Call - Duration of single invocation 17.36 ns 18.94 ns 1.09 0.08 False
Call - Duration of single invocation 17.31 ns 18.91 ns 1.09 0.06 False
Call - Duration of single invocation 17.48 ns 18.94 ns 1.08 0.06 False
Call - Duration of single invocation 17.30 ns 18.92 ns 1.09 0.07 False
Call - Duration of single invocation 17.26 ns 18.96 ns 1.10 0.07 False
Call - Duration of single invocation 17.29 ns 18.94 ns 1.10 0.08 False
Call - Duration of single invocation 17.31 ns 19.19 ns 1.11 0.11 False
Call - Duration of single invocation 17.43 ns 18.93 ns 1.09 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassInterface*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.979639080371733 > 18.260663557395375.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.001480768187882 (T) = (0 -19.050501127962782) / Math.Sqrt((0.6490263136820018 / (19)) + (0.10628566955264859 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0771230960836176 = (17.68646610329845 - 19.050501127962782) / 17.68646610329845 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.90923886393412 > 18.151829733800383.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.252031368834686 (T) = (0 -19.045570408406544) / Math.Sqrt((0.07937162257466379 / (19)) + (0.1010786192230382 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08688095822281922 = (17.523142957208798 - 19.045570408406544) / 17.523142957208798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.40)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.893571862296746 > 18.307941402107964.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -26.226476648657375 (T) = (0 -19.066267303692758) / Math.Sqrt((0.035161716605874214 / (19)) + (0.0558378319711709 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09324605738523484 = (17.44005128113098 - 19.066267303692758) / 17.44005128113098 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.90)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.936114507020754 > 18.16149588372402.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.194923024787723 (T) = (0 -19.117543407060246) / Math.Sqrt((0.01816221185100382 / (19)) + (0.12495852894404294 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09807066355310742 = (17.41012126232406 - 19.117543407060246) / 17.41012126232406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.70)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.909733280705577 > 18.167336098214793.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.949613957549088 (T) = (0 -19.073857849605062) / Math.Sqrt((0.10878836929157192 / (18)) + (0.11416126908457651 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.08495246514937951 = (17.580362700018306 - 19.073857849605062) / 17.580362700018306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.80)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.93562496697154 > 18.253819894172203.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.637329573688787 (T) = (0 -19.086584259644823) / Math.Sqrt((0.10291725886274361 / (19)) + (0.2405222457996731 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.08606555624552513 = (17.574062771704316 - 19.086584259644823) / 17.574062771704316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.20)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.917412747758377 > 18.15797294639591.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -25.164657409770047 (T) = (0 -19.065069450570014) / Math.Sqrt((0.03055176618745903 / (19)) + (0.07385973960100951 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09410038145767229 = (17.42533845493188 - 19.065069450570014) / 17.42533845493188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 1.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.96390771113831 > 18.130862351662003.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.714000150982407 (T) = (0 -19.03360570254727) / Math.Sqrt((0.08037252623027284 / (19)) + (0.12225278717087437 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08863327998202435 = (17.483946203501656 - 19.03360570254727) / 17.483946203501656 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.30)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.938427170868348 > 18.152896718085042.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.448598006708938 (T) = (0 -19.17686993112911) / Math.Sqrt((0.026014128105952702 / (19)) + (0.2424214630808787 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10024413875238632 = (17.429649707450004 - 19.17686993112911) / 17.429649707450004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.50)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.187651177031768 > 18.19223794014214.
IsChangePoint: Marked as a change because one of 7/23/2022 5:59:35 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.690948389577375 (T) = (0 -19.15686251862837) / Math.Sqrt((0.056358500538844315 / (19)) + (0.18748870382844304 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09983774368509717 = (17.417898802455824 - 19.15686251862837) / 17.417898802455824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.60)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.930478419893596 > 18.162256876324566.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.363830777846335 (T) = (0 -19.084854760909327) / Math.Sqrt((0.06513009338373435 / (19)) + (0.17468777915868522 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09206274759515193 = (17.47596903468814 - 19.084854760909327) / 17.47596903468814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 697.08 μs 756.79 μs 1.09 0.07 False
Queue - Duration of single invocation 25.64 μs 27.79 μs 1.08 0.07 False
SortedSet - Duration of single invocation 593.05 μs 640.82 μs 1.08 0.05 False
ImmutableSortedDictionary - Duration of single invocation 1.87 ms 2.01 ms 1.08 0.02 False
Stack - Duration of single invocation 14.98 μs 15.92 μs 1.06 0.07 False
LinkedList - Duration of single invocation 61.26 μs 69.69 μs 1.14 0.05 False
ImmutableList - Duration of single invocation 891.78 μs 998.40 μs 1.12 0.04 False
ImmutableSortedSet - Duration of single invocation 1.63 ms 1.81 ms 1.11 0.02 False
Span - Duration of single invocation 4.77 μs 5.30 μs 1.11 0.04 False
ICollection - Duration of single invocation 17.73 μs 19.12 μs 1.08 0.06 False
ImmutableHashSet - Duration of single invocation 1.91 ms 2.03 ms 1.06 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 756.7868059895834 > 735.1741498749382.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/14/2022 1:19:37 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.789668442961416 (T) = (0 -767256.5925449238) / Math.Sqrt((169736856.29368117 / (19)) + (149945645.2094993 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08435283213458218 = (707570.9767221757 - 767256.5925449238) / 707570.9767221757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.78818328990849 > 26.447697772388604.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.747848772103504 (T) = (0 -28437.87193632658) / Math.Sqrt((29200.286970014477 / (19)) + (2011165.507095826 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12599058608152783 = (25255.870064856408 - 28437.87193632658) / 25255.870064856408 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.SortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 640.8242369791666 > 613.0227922685186.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 1:45:10 PM, 10/14/2022 1:19:37 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.75026004375819 (T) = (0 -646710.9942059482) / Math.Sqrt((119167552.8438176 / (19)) + (99567138.56974109 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09998002069403872 = (587929.7642132647 - 646710.9942059482) / 587929.7642132647 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0077542020089285 > 1.9875589947743058.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.19989477732827 (T) = (0 -2052422.5145176162) / Math.Sqrt((381335133.8975823 / (19)) + (880172244.4859207 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0841592575798904 = (1893100.5755548563 - 2052422.5145176162) / 1893100.5755548563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.923353760683762 > 15.693282301090356.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.887427666685914 (T) = (0 -15963.311572113302) / Math.Sqrt((33572.82016046577 / (19)) + (80636.22598644286 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06304182346536359 = (15016.635488597429 - 15963.311572113302) / 15016.635488597429 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.68886753205129 > 64.39559284532046.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.049925060495614 (T) = (0 -68904.5024186602) / Math.Sqrt((303116.8593202597 / (19)) + (2796340.6562123145 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12221757592669429 = (61400.30587362783 - 68904.5024186602) / 61400.30587362783 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 998.39900625 > 934.8924960590278.
IsChangePoint: Marked as a change because one of 9/29/2022 5:24:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -28.77792033720846 (T) = (0 -996235.2671735763) / Math.Sqrt((64295135.294704646 / (19)) + (296436712.3127395 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12104097559931518 = (888669.8067757809 - 996235.2671735763) / 888669.8067757809 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8062485436507936 > 1.7140671130625.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.523001740293505 (T) = (0 -1804643.617726469) / Math.Sqrt((286892978.8668439 / (19)) + (763105895.0892576 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.0938933869346888 = (1649743.6032440478 - 1804643.617726469) / 1649743.6032440478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.296210917786781 > 5.0071976524323265.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.969977173826333 (T) = (0 -5278.1145791420195) / Math.Sqrt((322.73411473372306 / (19)) + (29427.089923721764 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10972820595798959 = (4756.222785727617 - 5278.1145791420195) / 4756.222785727617 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.119921665645425 > 18.5747303296051.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.937490196024406 (T) = (0 -19196.373505312247) / Math.Sqrt((140802.69014774266 / (19)) + (70738.21182607372 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07202159475923899 = (17906.704117861995 - 19196.373505312247) / 17906.704117861995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ImmutableHashSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0303119658854167 > 2.011211455588942.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.925202945036688 (T) = (0 -2047933.0359734765) / Math.Sqrt((634669921.8925105 / (19)) + (901632298.0965832 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.058284934713681034 = (1935143.3331398077 - 2047933.0359734765) / 1935143.3331398077 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableStack - Duration of single invocation 46.32 μs 49.31 μs 1.06 0.08 False
LinkedList - Duration of single invocation 64.27 μs 69.00 μs 1.07 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ImmutableStack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.31429598364094 > 49.19510350447496.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 10/12/2022 11:17:15 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.273223642298405 (T) = (0 -50045.254625899535) / Math.Sqrt((2233289.1383450353 / (19)) + (631191.4320185215 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0575239428019742 = (47323.04641094137 - 50045.254625899535) / 47323.04641094137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;String&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.99530657467533 > 67.23777360153404.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.66216221148954 (T) = (0 -68376.91477571454) / Math.Sqrt((2616431.1941456213 / (19)) + (393363.79444080463 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.05835980004404478 = (64606.49277577338 - 68376.91477571454) / 64606.49277577338 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateViewBetween - Duration of single invocation 44.47 μs 47.57 μs 1.07 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.57027003693715 > 45.78626177781086.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.20782567049513 (T) = (0 -48405.30668638125) / Math.Sqrt((429759.2668518213 / (19)) + (743281.3072981632 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10200895468292762 = (43924.60377085459 - 48405.30668638125) / 43924.60377085459 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinkedList - Duration of single invocation 97.31 μs 104.84 μs 1.08 0.07 False
Queue - Duration of single invocation 52.25 μs 56.03 μs 1.07 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<String>.LinkedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 104.8444021602349 > 101.68310390209793.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.791253922310457 (T) = (0 -105297.07090744884) / Math.Sqrt((794399.1167506711 / (19)) + (3119215.2085851906 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0881450057079039 = (96767.49914313741 - 105297.07090744884) / 96767.49914313741 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndRemove&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.031731269578835 > 54.512353137698504.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.597152436177979 (T) = (0 -56578.36210859971) / Math.Sqrt((2979893.5961790127 / (19)) + (2224480.192690145 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.089862671871053 = (51913.294737828925 - 56578.36210859971) / 51913.294737828925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 122.57 ns 144.14 ns 1.18 0.24 False
TryParse - Duration of single invocation 176.23 ns 191.27 ns 1.09 0.25 False
Parse - Duration of single invocation 114.98 ns 126.43 ns 1.10 0.28 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.TryParse(value: "0")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.14233488068646 > 128.89985644846917.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.649203665407091 (T) = (0 -133.8514527822057) / Math.Sqrt((7.795853095445622 / (19)) + (21.526334367976453 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10436693146052413 = (121.20197460565693 - 133.8514527822057) / 121.20197460565693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 191.27487533108703 > 189.93584800258608.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.244356365534258 (T) = (0 -194.84914629428823) / Math.Sqrt((41.59389460191368 / (22)) + (55.36283457307807 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.05767817063512602 = (184.22347336267998 - 194.84914629428823) / 184.22347336267998 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.Parse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.42547252319896 > 121.7578656718836.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.693316124936466 (T) = (0 -131.4702415717744) / Math.Sqrt((26.89481414576246 / (19)) + (55.46082734367539 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1035384961251548 = (119.13516568149161 - 131.4702415717744) / 119.13516568149161 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CtorDefaultSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 149.51 ns 159.47 ns 1.07 0.15 False
SortedList - Duration of single invocation 63.08 ns 71.07 ns 1.13 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<Int32>.SortedDictionary

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 159.4668202250548 > 156.88162700972748.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.115096438455875 (T) = (0 -165.07187722267045) / Math.Sqrt((9.194681535076752 / (19)) + (28.64732202712024 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0901102717844508 = (151.42676983719898 - 165.07187722267045) / 151.42676983719898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;Int32&gt;.SortedList

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.06868959465568 > 66.23444683985477.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.002041391123068 (T) = (0 -68.72557524340907) / Math.Sqrt((2.7569387393692555 / (19)) + (4.237634523239528 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07679416207440785 = (63.824245769508586 - 68.72557524340907) / 63.824245769508586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Threading.Tests.Perf_EventWaitHandle

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Set_Reset - Duration of single invocation 1.10 μs 1.19 μs 1.08 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_EventWaitHandle*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_EventWaitHandle.Set_Reset

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.186579002181058 > 1.1595998043334055.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.30968717326082 (T) = (0 -1192.809336729159) / Math.Sqrt((171.99384471769739 / (19)) + (419.8188234467528 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08139833887367594 = (1103.0249389614585 - 1192.809336729159) / 1103.0249389614585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPinnableReference - Duration of single invocation 13.85 ns 16.73 ns 1.21 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.GetPinnableReference

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.72505090660621 > 14.57826002212999.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -28.803181796807085 (T) = (0 -16.780087224275903) / Math.Sqrt((0.08269740100193872 / (19)) + (0.13339947561959187 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.19602248974475772 = (14.029909444141747 - 16.780087224275903) / 14.029909444141747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.IO.Tests.BinaryWriterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBool - Duration of single invocation 29.63 ns 33.60 ns 1.13 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterTests.WriteBool

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.602847043481724 > 31.0898730561684.
IsChangePoint: Marked as a change because one of 9/16/2022 12:08:08 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.41716901172677 (T) = (0 -31.711031255188537) / Math.Sqrt((0.9269588400726102 / (19)) + (0.5067452191184012 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.055226068666113695 = (30.051410021810494 - 31.711031255188537) / 30.051410021810494 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Lazy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ValueFromAlreadyInitialized - Duration of single invocation 12.69 ns 16.55 ns 1.30 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Lazy*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Lazy.ValueFromAlreadyInitialized

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.54519856050956 > 13.274726236782598.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.846197604425729 (T) = (0 -16.321782974383975) / Math.Sqrt((0.6928645289938868 / (19)) + (0.4042561865893664 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.25821362174074014 = (12.972187466705986 - 16.321782974383975) / 12.972187466705986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in V8.Crypto.Support

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Bench - Duration of single invocation 100.30 ms 113.67 ms 1.13 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'V8.Crypto.Support*'

Payloads

Baseline
Compare

Histogram

V8.Crypto.Support.Bench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.67329176666667 > 104.32817917173078.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.787264829470146 (T) = (0 -114004097.27431318) / Math.Sqrt((4723188624732.941 / (19)) + (5541029687035.574 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13874077770128596 = (100114178.3158473 - 114004097.27431318) / 100114178.3158473 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 2.49 ms 2.71 ms 1.09 0.01 False
ReadLineAsync - Duration of single invocation 716.68 μs 762.78 μs 1.06 0.01 False
ReadLineAsync - Duration of single invocation 8.83 ms 9.74 ms 1.10 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 8])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.711054086805555 > 2.6174235957291665.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/7/2022 4:10:19 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.840349887370149 (T) = (0 -2672241.7371731508) / Math.Sqrt((3077471936.2237797 / (19)) + (923153090.9520189 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08405685191681812 = (2465038.3717866093 - 2672241.7371731508) / 2465038.3717866093 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 762.7818321428574 > 752.5123748863639.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.74134441084565 (T) = (0 -765054.2076904102) / Math.Sqrt((236496357.04583675 / (18)) + (86216044.38754748 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.07163760351988402 = (713911.3121614296 - 765054.2076904102) / 713911.3121614296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.743396605714286 > 9.224708142421651.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/7/2022 4:10:19 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.796458239257252 (T) = (0 -9541471.94923662) / Math.Sqrt((143104331577.04953 / (19)) + (23182396060.97007 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10360060173928072 = (8645765.446484178 - 9541471.94923662) / 8645765.446484178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseHex - Duration of single invocation 112.87 ns 127.13 ns 1.13 0.18 False
ParseSpan - Duration of single invocation 131.14 ns 148.54 ns 1.13 0.29 False
TryParseHex - Duration of single invocation 198.90 ns 236.59 ns 1.19 0.18 False
TryParse - Duration of single invocation 96.13 ns 106.21 ns 1.10 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "0")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.13105282012098 > 119.7126819567466.
IsChangePoint: Marked as a change because one of 7/31/2022 3:07:22 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.036092363565466 (T) = (0 -128.53006979701564) / Math.Sqrt((5.370264941428003 / (19)) + (7.928631530076499 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12537339033546904 = (114.21104399731841 - 128.53006979701564) / 114.21104399731841 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ParseSpan(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.53506273604043 > 136.887216181471.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.95006388521627 (T) = (0 -147.88678926506114) / Math.Sqrt((8.1209128810437 / (19)) + (13.01806308489591 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12141591888629577 = (131.87505792849004 - 147.88678926506114) / 131.87505792849004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryParseHex(value: "3039")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.594611536347 > 209.60885800688294.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.861003992815819 (T) = (0 -220.26073363705763) / Math.Sqrt((20.08130392084678 / (19)) + (36.50271878034806 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0901984839558231 = (202.0372775036651 - 220.26073363705763) / 202.0372775036651 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryParse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.21058799740779 > 99.6025143905054.
IsChangePoint: Marked as a change because one of 10/11/2022 10:45:23 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.235007722358368 (T) = (0 -107.41671612441186) / Math.Sqrt((7.400688135681609 / (19)) + (11.319035540063602 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11294036756224149 = (96.5161470058768 - 107.41671612441186) / 96.5161470058768 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 30.57 μs 32.16 μs 1.05 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 1000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.158353567761814 > 31.917084569846676.
IsChangePoint: Marked as a change because one of 7/25/2022 9:08:25 PM, 7/26/2022 9:42:12 AM, 10/11/2022 8:42:28 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.211046483636512 (T) = (0 -32585.990903366925) / Math.Sqrt((374190.84583738993 / (19)) + (365084.0222906933 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.06056911299110558 = (30725.00462649265 - 32585.990903366925) / 30725.00462649265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchstone.BenchI.Ackermann

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 25.72 μs 27.15 μs 1.06 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Ackermann*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Ackermann.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.151303132233796 > 27.03550128599534.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.969022640386868 (T) = (0 -27435.298395424437) / Math.Sqrt((60768.19750874508 / (19)) + (119567.29832059034 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.059998330584079315 = (25882.397739538952 - 27435.298395424437) / 25882.397739538952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchstone.BenchI.BenchE

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 4.12 secs 4.36 secs 1.06 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.BenchE.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.358953525785714 > 4.32997013697.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.50809563758034 (T) = (0 -4361880398.423365) / Math.Sqrt((1265913313662783 / (19)) + (1878587048381949.2 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05453818234676875 = (4136294419.151745 - 4361880398.423365) / 4136294419.151745 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Bag - Duration of single invocation 58.04 ns 74.07 ns 1.28 0.22 False
Queue - Duration of single invocation 69.29 ns 75.70 ns 1.09 0.20 False
Queue - Duration of single invocation 53.54 ns 63.29 ns 1.18 0.18 False
Bag - Duration of single invocation 66.23 ns 74.44 ns 1.12 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<String>.Bag(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.06746456999811 > 61.86580437088717.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.171400570591882 (T) = (0 -70.61159031859843) / Math.Sqrt((9.424049889466788 / (19)) + (7.198126775853886 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.17589488583854826 = (60.04923668686955 - 70.61159031859843) / 60.04923668686955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;String&gt;.Queue(Size: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.70387269434481 > 72.66559112650607.
IsChangePoint: Marked as a change because one of 11/15/2022 2:18:08 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.178774844426712 (T) = (0 -75.31627574076019) / Math.Sqrt((2.990720750302793 / (17)) + (4.636986602240342 / (30))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (30) - 2, .025) and -0.07533922841632294 = (70.03955007916917 - 75.31627574076019) / 70.03955007916917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.28958890691301 > 57.36240591672616.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.8954654011547944 (T) = (0 -61.15568929862149) / Math.Sqrt((3.989449463042843 / (19)) + (8.840442096203681 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10327470756636405 = (55.431062526141396 - 61.15568929862149) / 55.431062526141396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;String&gt;.Bag(Size: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.44491118218323 > 68.47317173623517.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 1:45:10 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.945209921617392 (T) = (0 -74.90405139700795) / Math.Sqrt((4.714459721486882 / (19)) + (5.068905040100738 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.15036345952237784 = (65.11337853873378 - 74.90405139700795) / 65.11337853873378 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 365.38 ms 398.28 ms 1.09 0.06 False
Count - Duration of single invocation 156.50 ms 175.31 ms 1.12 0.08 False
Count - Duration of single invocation 155.95 ms 171.25 ms 1.10 0.09 False
Count - Duration of single invocation 531.37 ms 582.20 ms 1.10 0.04 False
Count - Duration of single invocation 153.77 ms 171.67 ms 1.12 0.06 False
Count - Duration of single invocation 157.03 ms 171.80 ms 1.09 0.07 False
Count - Duration of single invocation 15.38 ms 16.23 ms 1.06 0.02 False
Count - Duration of single invocation 22.75 ms 24.31 ms 1.07 0.03 False
Count - Duration of single invocation 136.29 ms 145.63 ms 1.07 0.05 False
Count - Duration of single invocation 114.36 ms 122.36 ms 1.07 0.03 False
Count - Duration of single invocation 376.33 ms 406.76 ms 1.08 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 398.2837118 > 392.68883332.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.923190990832778 (T) = (0 -395081774.7465986) / Math.Sqrt((35263388217096.54 / (19)) + (83533805576941.69 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05848452228492949 = (373252292.71538466 - 395081774.7465986) / 373252292.71538466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 175.3123287142857 > 164.47543040250002.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.222242987095633 (T) = (0 -172627654.17261904) / Math.Sqrt((5639870144026.941 / (19)) + (21832733957513.703 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10070767668823577 = (156833333.52594948 - 172627654.17261904) / 156833333.52594948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes\\s+\\w+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 171.2485812666667 > 164.30732703000004.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.781678370356637 (T) = (0 -172592153.42832235) / Math.Sqrt((21510077389052.207 / (22)) + (8928987515042.713 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.08562246445108042 = (158979902.38769564 - 172592153.42832235) / 158979902.38769564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{Ll}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 582.2016417333333 > 557.3840864415386.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.617104688096152 (T) = (0 -578517234.9681056) / Math.Sqrt((131211102053350.58 / (19)) + (68017434156857 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08989307588487172 = (530801826.13181025 - 578517234.9681056) / 530801826.13181025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 171.6662957857143 > 163.31833105000004.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.18341362227131 (T) = (0 -171131915.61718994) / Math.Sqrt((6331712858483.528 / (19)) + (9647136113698.8 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09586222368448392 = (156161889.6231444 - 171131915.61718994) / 156161889.6231444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes\\s+\\w+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 171.80008415384614 > 163.418656835.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.953751717787918 (T) = (0 -172439308.24406454) / Math.Sqrt((6902182063203.46 / (19)) + (16761091482541.934 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.09412151205194463 = (157605262.61901867 - 172439308.24406454) / 157605262.61901867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\\s+\\w+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.23193536888889 > 16.162280489754806.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.261058663820583 (T) = (0 -16290630.188125404) / Math.Sqrt((33960076336.696674 / (19)) + (70772381149.96213 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0564930766033331 = (15419533.311567381 - 16290630.188125404) / 15419533.311567381 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\\n]*", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.305926371428573 > 24.014246313227275.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.406010555607724 (T) = (0 -24269351.15456169) / Math.Sqrt((114877007489.41821 / (19)) + (112980573426.03778 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05410186292605036 = (23023724.753880154 - 24269351.15456169) / 23023724.753880154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.6295852142857 > 142.9774670625.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.525234857804934 (T) = (0 -145881499.49972525) / Math.Sqrt((8353282949617.382 / (19)) + (11079073704017.773 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05642522930336402 = (138089753.49436092 - 145881499.49972525) / 138089753.49436092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.36353414285715 > 120.58763738750001.
IsChangePoint: Marked as a change because one of 10/13/2022 5:34:08 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.203403228034187 (T) = (0 -123570319.953385) / Math.Sqrt((1520190044840.1577 / (19)) + (5683837635185.924 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07003602438260832 = (115482392.309813 - 123570319.953385) / 115482392.309813 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 406.7564412857143 > 389.00024362500005.
IsChangePoint: Marked as a change because one of 10/13/2022 5:34:08 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.48687180100239 (T) = (0 -399817934.8275706) / Math.Sqrt((17663163319989.434 / (19)) + (22472361698883.05 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07994086408821975 = (370222063.19151723 - 399817934.8275706) / 370222063.19151723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Logging.ScopesOverheadBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NotFiltered - Duration of single invocation 218.82 ns 250.38 ns 1.14 0.16 False
NotFiltered - Duration of single invocation 214.95 ns 236.69 ns 1.10 0.14 False
FilteredByLevel - Duration of single invocation 66.17 ns 71.42 ns 1.08 0.26 False
NotFiltered - Duration of single invocation 215.08 ns 236.25 ns 1.10 0.12 False
NotFiltered_InsideScope - Duration of single invocation 255.64 ns 284.61 ns 1.11 0.10 False
FilteredByLevel_InsideScope - Duration of single invocation 100.60 ns 109.81 ns 1.09 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.ScopesOverheadBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger: True, CaptureScopes: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.3824516107539 > 230.1678637840152.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.572950779792727 (T) = (0 -238.9674012047657) / Math.Sqrt((55.927732745641066 / (19)) + (34.79298194897738 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07921514137383032 = (221.42702788673122 - 238.9674012047657) / 221.42702788673122 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger: False, CaptureScopes: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.6941694943461 > 228.0413262044278.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.91902201368578 (T) = (0 -237.04458863630558) / Math.Sqrt((37.776699664385994 / (19)) + (32.02250323663089 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07133752149830785 = (221.26041875653655 - 237.04458863630558) / 221.26041875653655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: False, CaptureScopes: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.4164406411937 > 69.89884973485636.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/9/2022 12:33:05 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.808650645504487 (T) = (0 -72.74059110863482) / Math.Sqrt((3.01555739387342 / (15)) + (8.73561983214378 / (32))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (32) - 2, .025) and -0.07980800633536014 = (67.36437466832737 - 72.74059110863482) / 67.36437466832737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger: True, CaptureScopes: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.2540713311963 > 226.80926358090852.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.273779333674872 (T) = (0 -234.98610331951733) / Math.Sqrt((17.938582849419383 / (19)) + (19.49262478155758 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07798806712301651 = (217.98581124061874 - 234.98610331951733) / 217.98581124061874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered_InsideScope(HasISupportLoggingScopeLogger: True, CaptureScopes: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 284.61331552611415 > 267.34333813050756.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.305812924209285 (T) = (0 -275.83678662353316) / Math.Sqrt((24.873726505196935 / (19)) + (17.117260936633134 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08330253535539431 = (254.6258110002859 - 275.83678662353316) / 254.6258110002859 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel_InsideScope(HasISupportLoggingScopeLogger: False, CaptureScopes: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.80766737069938 > 107.00956985632382.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.602512772455187 (T) = (0 -112.55946965009475) / Math.Sqrt((19.493188975754595 / (19)) + (27.93078780909551 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09103672466582509 = (103.16744350156566 - 112.55946965009475) / 103.16744350156566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TransformByQuaternionBenchmark - Duration of single invocation 89.19 ns 99.55 ns 1.12 0.16 False
TransformVector2ByMatrix4x4Benchmark - Duration of single invocation 41.02 ns 49.40 ns 1.20 0.25 False
TransformByMatrix4x4Benchmark - Duration of single invocation 63.22 ns 74.66 ns 1.18 0.19 False
TransformVector3ByMatrix4x4Benchmark - Duration of single invocation 62.01 ns 74.26 ns 1.20 0.19 False
LengthSquaredBenchmark - Duration of single invocation 17.96 ns 20.60 ns 1.15 0.26 False
LengthBenchmark - Duration of single invocation 19.82 ns 22.28 ns 1.12 0.13 False
DotBenchmark - Duration of single invocation 16.96 ns 19.40 ns 1.14 0.29 False
TransformVector3ByQuaternionBenchmark - Duration of single invocation 89.10 ns 98.71 ns 1.11 0.20 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.5473895586634 > 94.46898148314568.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.786045195583782 (T) = (0 -98.4006440217555) / Math.Sqrt((8.429026689887575 / (19)) + (7.658241721526671 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08179829424758574 = (90.96025067241882 - 98.4006440217555) / 90.96025067241882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.40268466463353 > 43.10112925016626.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.313739213814591 (T) = (0 -49.71672218898364) / Math.Sqrt((3.168205075552063 / (19)) + (3.833132472612559 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1885627765607162 = (41.82927748490189 - 49.71672218898364) / 41.82927748490189 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.66238166159673 > 65.83000803384076.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.202517632724167 (T) = (0 -75.08708770393663) / Math.Sqrt((3.51831224355624 / (19)) + (3.9951162484241483 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.17316795850101166 = (64.00369798701068 - 75.08708770393663) / 64.00369798701068 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.25810639221868 > 65.44389440732644.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.410442713606326 (T) = (0 -75.12171721122563) / Math.Sqrt((1.3910205438678043 / (19)) + (8.989213466892455 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1818679916676984 = (63.56185101960806 - 75.12171721122563) / 63.56185101960806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.LengthSquaredBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.599029858435802 > 18.712326343796807.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -4.39224705226615 (T) = (0 -20.123590262750458) / Math.Sqrt((2.3854222711098347 / (22)) + (0.8711500147294573 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.0900563563530699 = (18.461054922037825 - 20.123590262750458) / 18.461054922037825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.LengthBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.279675864718907 > 21.060122440399407.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.759791020529698 (T) = (0 -22.535820865005793) / Math.Sqrt((1.7018662561963456 / (19)) + (1.793511150585665 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11132896373451531 = (20.278262873016743 - 22.535820865005793) / 20.278262873016743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.DotBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.40374424233421 > 18.254825370107802.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -4.863408678466875 (T) = (0 -19.78838257106134) / Math.Sqrt((1.8804027746844862 / (19)) + (2.0326015217324738 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11333563073607072 = (17.773959644117785 - 19.78838257106134) / 17.773959644117785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.71055949444654 > 94.83104873972262.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.9649674585726675 (T) = (0 -97.99082158370084) / Math.Sqrt((11.094372872111471 / (19)) + (6.166663244532706 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0786206247967371 = (90.84827355509442 - 97.99082158370084) / 90.84827355509442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 110.07 ns 122.74 ns 1.12 0.27 False
ParseSpan - Duration of single invocation 189.65 ns 206.69 ns 1.09 0.26 False
ParseSpan - Duration of single invocation 133.23 ns 142.54 ns 1.07 0.32 False
Parse - Duration of single invocation 173.39 ns 184.48 ns 1.06 0.19 False
TryParse - Duration of single invocation 95.96 ns 108.28 ns 1.13 0.23 False
TryParseHex - Duration of single invocation 112.13 ns 128.16 ns 1.14 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.Parse(value: "0")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.73542566862157 > 117.61057475491921.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.010957578794889 (T) = (0 -123.83962155566714) / Math.Sqrt((19.499256997149782 / (19)) + (11.458846658660569 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.08449646300119593 = (114.190894835155 - 123.83962155566714) / 114.190894835155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ParseSpan(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 206.6899633777526 > 198.91519113210103.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 9:30:44 PM, 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.36684895726938 (T) = (0 -207.17733067722133) / Math.Sqrt((42.474094356258256 / (18)) + (25.066075022085496 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.07818274044049509 = (192.15418955100168 - 207.17733067722133) / 192.15418955100168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ParseSpan(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.5421081679299 > 137.33703062716395.
IsChangePoint: Marked as a change because one of 11/15/2022 2:18:08 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.933141797130805 (T) = (0 -150.26129260718687) / Math.Sqrt((16.76128498650943 / (17)) + (48.75761771654429 / (30))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (30) - 2, .025) and -0.13324215100530468 = (132.59416133955955 - 150.26129260718687) / 132.59416133955955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 184.48278488861916 > 181.9853777419199.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.079948942512281 (T) = (0 -186.14837981813326) / Math.Sqrt((7.817967178568141 / (19)) + (24.86296764242448 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0727897825010821 = (173.5180394654304 - 186.14837981813326) / 173.5180394654304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.27950566726184 > 100.48802608517767.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.553794817273216 (T) = (0 -107.07293195493601) / Math.Sqrt((13.11995727869894 / (19)) + (10.125613914446934 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10074472218084704 = (97.2731731502633 - 107.07293195493601) / 97.2731731502633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParseHex(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.16299410800488 > 118.33868737469686.
IsChangePoint: Marked as a change because one of 8/2/2022 2:48:04 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.856979085257855 (T) = (0 -128.3826486135304) / Math.Sqrt((5.11739292777155 / (19)) + (9.158259177731436 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12794681312043657 = (113.8197715709334 - 128.3826486135304) / 113.8197715709334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseDecimal - Duration of single invocation 567.97 ns 634.86 ns 1.12 0.05 False
TryParseInt16 - Duration of single invocation 54.66 ns 58.76 ns 1.07 0.22 False
TryParseSingle - Duration of single invocation 829.63 ns 959.12 ns 1.16 0.03 False
TryParseSingle - Duration of single invocation 392.42 ns 455.12 ns 1.16 0.14 False
TryParseSingle - Duration of single invocation 853.16 ns 957.57 ns 1.12 0.06 False
TryParseDouble - Duration of single invocation 995.90 ns 1.12 μs 1.13 0.04 False
TryParseInt32 - Duration of single invocation 55.71 ns 59.53 ns 1.07 0.21 False
TryParseDouble - Duration of single invocation 397.90 ns 431.14 ns 1.08 0.13 False
TryParseDouble - Duration of single invocation 976.43 ns 1.08 μs 1.11 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 634.8604717400051 > 604.5640130992753.
IsChangePoint: Marked as a change because one of 9/29/2022 2:35:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.200632685186763 (T) = (0 -640.2260787892654) / Math.Sqrt((174.44509345762768 / (19)) + (299.83907936678384 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10978108169503686 = (576.894028335218 - 640.2260787892654) / 576.894028335218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.75776600302042 > 57.47426814793439.
IsChangePoint: Marked as a change because one of 8/2/2022 8:20:46 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.249773867896194 (T) = (0 -59.63869184152371) / Math.Sqrt((3.7421138741659408 / (19)) + (3.109884379330122 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07234719473299489 = (55.61509568398062 - 59.63869184152371) / 55.61509568398062 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 959.1199568105277 > 873.1508647619955.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.20741366165905 (T) = (0 -943.819211666039) / Math.Sqrt((145.71723264419495 / (19)) + (960.0436131746028 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12515149712967305 = (838.8374490668828 - 943.819211666039) / 838.8374490668828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 455.1162556676575 > 419.39871081884013.
IsChangePoint: Marked as a change because one of 10/1/2022 9:48:16 AM, 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.586945257364627 (T) = (0 -439.1389731001298) / Math.Sqrt((224.31245221398603 / (22)) + (77.9597744115607 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.09645396276867607 = (400.5083551262398 - 439.1389731001298) / 400.5083551262398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 957.5685336327782 > 877.6416320019056.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.391384381107446 (T) = (0 -958.6383427153762) / Math.Sqrt((142.693395495529 / (19)) + (776.3989996907519 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13646458336621256 = (843.5268082669903 - 958.6383427153762) / 843.5268082669903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: -1.7976931348623157e+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1218273462360346 > 1.0626962914740894.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.443990075795488 (T) = (0 -1092.0892597481427) / Math.Sqrt((319.24739801271954 / (19)) + (801.9958716948689 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09049799304871531 = (1001.4592110297963 - 1092.0892597481427) / 1001.4592110297963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.531650516533475 > 59.47986544834788.
IsChangePoint: Marked as a change because one of 10/1/2022 3:26:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.55013807794637 (T) = (0 -61.563042915819956) / Math.Sqrt((4.317824180387711 / (19)) + (8.655645118191147 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08450868714934165 = (56.765836590613176 - 61.563042915819956) / 56.765836590613176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 431.14134194616383 > 409.1740720275374.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.44792862993287 (T) = (0 -434.33424907654506) / Math.Sqrt((59.74088500993159 / (19)) + (172.80476575582378 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.1145385516253362 = (389.6987218998873 - 434.33424907654506) / 389.6987218998873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0849332730434784 > 1044.2101617563142.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.028481350198033 (T) = (0 -1080.3059577587424) / Math.Sqrt((238.71095839513043 / (19)) + (810.8478398150357 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09131823226397466 = (989.9091995536563 - 1080.3059577587424) / 989.9091995536563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FooObjCastIfIsa - Duration of single invocation 1.34 ms 1.42 ms 1.05 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.FooObjCastIfIsa

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4156448117424243 > 1.405436263828125.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.589165556693107 (T) = (0 -1428890.0443584688) / Math.Sqrt((412146450.5871861 / (19)) + (1016065274.8909818 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05980359431254953 = (1348259.2925959365 - 1428890.0443584688) / 1348259.2925959365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Next_int_int - Duration of single invocation 92.20 ns 98.06 ns 1.06 0.18 False
Next_int - Duration of single invocation 83.72 ns 89.53 ns 1.07 0.13 False
Next_int_int_unseeded - Duration of single invocation 151.62 ns 161.84 ns 1.07 0.19 False
NextSingle - Duration of single invocation 73.53 ns 84.60 ns 1.15 0.22 False
Next_long - Duration of single invocation 429.86 ns 486.20 ns 1.13 0.09 False
Next_long_long - Duration of single invocation 450.75 ns 514.90 ns 1.14 0.08 False
Next_int_unseeded - Duration of single invocation 139.90 ns 155.28 ns 1.11 0.15 False
NextDouble - Duration of single invocation 70.30 ns 81.65 ns 1.16 0.21 False
Next_unseeded - Duration of single invocation 53.75 ns 60.25 ns 1.12 0.24 False
NextBytes - Duration of single invocation 39.02 μs 41.84 μs 1.07 0.13 False
Next - Duration of single invocation 62.89 ns 66.73 ns 1.06 0.27 False
NextSingle_unseeded - Duration of single invocation 54.86 ns 60.36 ns 1.10 0.29 False
NextDouble_unseeded - Duration of single invocation 55.32 ns 60.56 ns 1.09 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.Next_int_int

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.0600242511069 > 93.77662798911446.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.698818210384323 (T) = (0 -98.32964140388947) / Math.Sqrt((6.2207337206107685 / (19)) + (2.9144114442937936 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10073193384183268 = (89.33114265222974 - 98.32964140388947) / 89.33114265222974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next_int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 89.52563301141831 > 85.9845657336774.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.801159066750161 (T) = (0 -90.41134088118642) / Math.Sqrt((11.510020799760097 / (19)) + (5.132288329601362 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10655935150058111 = (81.70491782350541 - 90.41134088118642) / 81.70491782350541 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next_int_int_unseeded

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 161.84013856357814 > 160.4021574258113.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/14/2022 3:33:41 AM, 11/15/2022 2:18:08 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.571319361406413 (T) = (0 -163.67374302425785) / Math.Sqrt((22.506240007657116 / (17)) + (11.144559133761163 / (30))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (30) - 2, .025) and -0.06409226072114667 = (153.81536833407134 - 163.67374302425785) / 153.81536833407134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.NextSingle

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.59991090275074 > 76.45371202806784.
IsChangePoint: Marked as a change because one of 10/12/2022 1:08:53 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.908791276000127 (T) = (0 -84.04808943856303) / Math.Sqrt((14.904052001433993 / (19)) + (4.412732041980669 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12921060454821048 = (74.43083610801733 - 84.04808943856303) / 74.43083610801733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next_long

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 486.19736734564145 > 452.27262076870335.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.46459515718854 (T) = (0 -484.6812681051603) / Math.Sqrt((78.17406949575482 / (19)) + (70.24582217013027 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12190989867018029 = (432.01443242426296 - 484.6812681051603) / 432.01443242426296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next_long_long

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 514.8993624019839 > 473.407117132906.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.84909658233125 (T) = (0 -517.2662086450622) / Math.Sqrt((65.32129202993308 / (19)) + (63.95560682927668 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.12498854243146966 = (459.796868265947 - 517.2662086450622) / 459.796868265947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next_int_unseeded

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.28191691953936 > 146.66642844330144.
IsChangePoint: Marked as a change because one of 10/12/2022 11:17:15 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.453667543395364 (T) = (0 -151.50601011468285) / Math.Sqrt((6.838524346869118 / (19)) + (10.909560951339238 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07003460738087706 = (141.58982248763337 - 151.50601011468285) / 141.58982248763337 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.NextDouble

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.64950394422596 > 75.78208272081297.
IsChangePoint: Marked as a change because one of 10/12/2022 1:08:53 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.9504734472966 (T) = (0 -82.8738559524041) / Math.Sqrt((2.5011352829137667 / (19)) + (3.440980848506044 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14618439699610175 = (72.304121544141 - 82.8738559524041) / 72.304121544141 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next_unseeded

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.24607026128059 > 56.47034999496284.
IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.30459265492066 (T) = (0 -60.58899606932606) / Math.Sqrt((2.800378947099983 / (19)) + (5.434211506556004 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09858525521041568 = (55.15183804075474 - 60.58899606932606) / 55.15183804075474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.NextBytes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.83545916491595 > 41.09069283903761.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.306986231509974 (T) = (0 -42106.29366856865) / Math.Sqrt((179705.13458016957 / (19)) + (1485689.0190789788 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07197625456656233 = (39279.129075105964 - 42106.29366856865) / 39279.129075105964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73251060712211 > 66.68611292837761.
IsChangePoint: Marked as a change because one of 11/15/2022 10:02:59 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.203483996879823 (T) = (0 -68.84980359710994) / Math.Sqrt((4.721699775902916 / (20)) + (6.00337376477018 / (27))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (27) - 2, .025) and -0.0539313617666323 = (65.32664848468099 - 68.84980359710994) / 65.32664848468099 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.NextSingle_unseeded

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.3566924026927 > 57.87259610277917.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.358573123271926 (T) = (0 -61.53040877773635) / Math.Sqrt((5.690861259798897 / (19)) + (2.5627053541328375 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09283456241031401 = (56.30349816355299 - 61.53040877773635) / 56.30349816355299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.NextDouble_unseeded

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.557278936532015 > 57.21914574464846.
IsChangePoint: Marked as a change because one of 10/14/2022 3:33:41 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.49165744470518 (T) = (0 -60.40187957928708) / Math.Sqrt((1.296586454465415 / (18)) + (4.080933515078747 / (28))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (28) - 2, .025) and -0.11636736111105458 = (54.105737666114365 - 60.40187957928708) / 54.105737666114365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ProducerConsumer - Duration of single invocation 4.21 μs 4.65 μs 1.10 0.05 False
SingleSerial - Duration of single invocation 1.97 μs 2.13 μs 1.08 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.6459910099999995 > 4.35229878525.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.517747423029267 (T) = (0 -4454.555944910888) / Math.Sqrt((2946.828532616083 / (19)) + (29226.728671385765 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07989194304939262 = (4125.00155555577 - 4454.555944910888) / 4125.00155555577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.128580970833333 > 2.0605110722.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/16/2022 9:45:14 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.3842561820337576 (T) = (0 -2120.932239921406) / Math.Sqrt((868.0227923982515 / (21)) + (18839.312621071724 / (26))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (26) - 2, .025) and -0.07556641398099248 = (1971.9212243446711 - 2120.932239921406) / 1971.9212243446711 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsBenchmark - Duration of single invocation 36.21 ns 39.30 ns 1.09 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.30350009179783 > 38.619971195156154.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.005558459325681 (T) = (0 -39.88774451278302) / Math.Sqrt((1.7190281419690157 / (19)) + (1.259162667940616 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06910433281413575 = (37.30949663985463 - 39.88774451278302) / 37.30949663985463 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 2.60 ms 2.76 ms 1.06 0.01 False
Queue - Duration of single invocation 30.52 μs 32.89 μs 1.08 0.08 False
LinkedList - Duration of single invocation 61.71 μs 73.54 μs 1.19 0.10 False
ImmutableQueue - Duration of single invocation 63.11 μs 66.79 μs 1.06 0.08 False
ImmutableList - Duration of single invocation 895.37 μs 1.02 ms 1.14 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.757886854166667 > 2.729151858359375.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.404579299157877 (T) = (0 -2779268.2359916004) / Math.Sqrt((921175673.2066078 / (19)) + (1450353263.9487958 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06282097050866821 = (2614991.9065498277 - 2779268.2359916004) / 2614991.9065498277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.88563736263736 > 31.6839045012669.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.529012083048322 (T) = (0 -33135.9586267863) / Math.Sqrt((49754.61794125388 / (19)) + (947539.8508126017 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09827920816851372 = (30170.796624697738 - 33135.9586267863) / 30170.796624697738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.5386568955399 > 65.10032954893728.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.724011011017616 (T) = (0 -69770.70192445176) / Math.Sqrt((829961.3897539332 / (19)) + (1784133.297071593 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11949159340429735 = (62323.56038716095 - 69770.70192445176) / 62323.56038716095 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.ImmutableQueue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.78672646011395 > 66.25361369812501.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.968189253288031 (T) = (0 -67471.12918428773) / Math.Sqrt((909638.7791347646 / (19)) + (866399.0271237259 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06641917004246602 = (63268.86376358083 - 67471.12918428773) / 63268.86376358083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0165342658854166 > 939.5444287239585.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.02843837972708 (T) = (0 -997642.1845049079) / Math.Sqrt((261069882.54291737 / (19)) + (109493605.69993013 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11258158001347751 = (896691.2650961045 - 997642.1845049079) / 896691.2650961045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetValue - Duration of single invocation 31.34 ns 34.61 ns 1.10 0.23 False
Equals_Object_Invalid - Duration of single invocation 12.40 ns 14.22 ns 1.15 0.33 False
TrimEnd - Duration of single invocation 273.85 ns 299.73 ns 1.09 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetValue

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.61107096556818 > 32.822204190285746.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.464247362479401 (T) = (0 -35.33639682372658) / Math.Sqrt((0.8085698966824038 / (19)) + (0.9923189175778757 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10927475656614977 = (31.85540517762549 - 35.33639682372658) / 31.85540517762549 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Invalid

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.224941803248218 > 13.471461717459507.
IsChangePoint: Marked as a change because one of 9/24/2022 7:43:48 AM, 9/24/2022 8:58:25 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -3.8498639252728113 (T) = (0 -14.114842530972865) / Math.Sqrt((0.8060263816833273 / (19)) + (0.2835026488222082 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06669377169552858 = (13.232328626553311 - 14.114842530972865) / 13.232328626553311 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 299.7280315280763 > 286.4801504201388.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.537120007562358 (T) = (0 -300.4150976509623) / Math.Sqrt((47.371780502091354 / (19)) + (55.7257789986036 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08022723944080694 = (278.10361253848396 - 300.4150976509623) / 278.10361253848396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.Add_Remove_SteadyState<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableQueue - Duration of single invocation 318.73 ns 344.39 ns 1.08 0.04 False
ConcurrentBag - Duration of single invocation 272.20 ns 286.42 ns 1.05 0.13 False
Queue - Duration of single invocation 79.16 ns 88.06 ns 1.11 0.08 False
ImmutableStack - Duration of single invocation 88.86 ns 97.46 ns 1.10 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Add_Remove_SteadyState<Int32>.ImmutableQueue(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 344.38876584216626 > 330.91280278029456.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -41.96296022528495 (T) = (0 -346.2066625102985) / Math.Sqrt((4.721797558351658 / (19)) + (8.9483191927028 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10054241131869643 = (314.5782106620183 - 346.2066625102985) / 314.5782106620183 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Add_Remove_SteadyState&lt;Int32&gt;.ConcurrentBag(Count: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 286.41663155185233 > 283.3044775063649.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.569724593243545 (T) = (0 -292.4022299943402) / Math.Sqrt((100.36921490103332 / (19)) + (41.5209369296209 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.062068059334850226 = (275.3140228861277 - 292.4022299943402) / 275.3140228861277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Add_Remove_SteadyState&lt;Int32&gt;.Queue(Count: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 88.06324518838905 > 83.53126906121415.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.36298761465103 (T) = (0 -88.55153042168072) / Math.Sqrt((3.828849163512494 / (19)) + (7.706967640001506 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10668533148390939 = (80.01509363365788 - 88.55153042168072) / 80.01509363365788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Add_Remove_SteadyState&lt;Int32&gt;.ImmutableStack(Count: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.4571853642923 > 93.21910131783285.
IsChangePoint: Marked as a change because one of 7/30/2022 2:52:48 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.101913677544186 (T) = (0 -98.8310936844479) / Math.Sqrt((23.237185166722064 / (19)) + (12.738323243480616 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08692866069095508 = (90.92693684387848 - 98.8310936844479) / 90.92693684387848 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in GuardedDevirtualization.ThreeClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 17.27 ns 19.35 ns 1.12 0.06 False
Call - Duration of single invocation 17.27 ns 19.03 ns 1.10 0.08 False
Call - Duration of single invocation 17.29 ns 18.92 ns 1.09 0.05 False
Call - Duration of single invocation 17.24 ns 19.28 ns 1.12 0.06 False
Call - Duration of single invocation 17.30 ns 18.92 ns 1.09 0.06 False
Call - Duration of single invocation 17.67 ns 18.91 ns 1.07 0.09 False
Call - Duration of single invocation 17.41 ns 18.91 ns 1.09 0.08 False
Call - Duration of single invocation 17.26 ns 19.06 ns 1.10 0.04 False
Call - Duration of single invocation 17.26 ns 18.96 ns 1.10 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassInterface*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.33)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.345442057177827 > 18.146716985298347.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.184195395106155 (T) = (0 -19.187565514782165) / Math.Sqrt((0.04473998232096833 / (19)) + (0.2664089078181655 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.10283688347975628 = (17.398371238945213 - 19.187565514782165) / 17.398371238945213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.030298244810748 > 18.11708561233283.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.147468633185092 (T) = (0 -19.08847729357651) / Math.Sqrt((0.07782253816766715 / (19)) + (0.17989131162035463 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09501058665176822 = (17.432230817003934 - 19.08847729357651) / 17.432230817003934 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.67)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.92491045491636 > 18.221583075435028.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.967143656468856 (T) = (0 -19.094596752326076) / Math.Sqrt((0.07191720065533644 / (19)) + (0.06743222061881111 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08967408835328508 = (17.523218140555972 - 19.094596752326076) / 17.523218140555972 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.67)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.277822388136293 > 18.298853195411574.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -28.448164592925774 (T) = (0 -19.01601039554944) / Math.Sqrt((0.027611030342722498 / (19)) + (0.04631229917068356 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09097814889849357 = (17.430239473401883 - 19.01601039554944) / 17.430239473401883 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=1.00 pD=0.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.918993293298584 > 18.153965824646814.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -29.70275185659211 (T) = (0 -19.00269465068425) / Math.Sqrt((0.018611928830270203 / (19)) + (0.056768776061017316 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09374935658273074 = (17.37390247255144 - 19.00269465068425) / 17.37390247255144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=1.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.91317320019724 > 18.142238939515615.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.338583686971793 (T) = (0 -19.06510183063612) / Math.Sqrt((0.1176566917147388 / (19)) + (0.12836604611752367 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.08540891798403097 = (17.56490251254469 - 19.06510183063612) / 17.56490251254469 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.91098049055829 > 18.272001456142345.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.97518749860851 (T) = (0 -19.14713490945393) / Math.Sqrt((0.06392973779393073 / (19)) + (0.1768478214510374 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09556592154204083 = (17.47693546592229 - 19.14713490945393) / 17.47693546592229 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.056604731812577 > 18.135657127727168.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.09310407752082 (T) = (0 -19.08050394240853) / Math.Sqrt((0.17384978158589837 / (19)) + (0.0632013065520584 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08564323392735639 = (17.57529853834585 - 19.08050394240853) / 17.57529853834585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.959865591890793 > 18.23656405329038.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.330496994961493 (T) = (0 -19.107200004509394) / Math.Sqrt((0.030248025129993212 / (19)) + (0.09393203425893926 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09836961362134754 = (17.395965590774637 - 19.107200004509394) / 17.395965590774637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Await_FromCompletedValueTaskSource - Duration of single invocation 560.33 ns 595.08 ns 1.06 0.04 False
CreateAndAwait_FromCompletedValueTaskSource_ConfigureAwait - Duration of single invocation 415.84 ns 450.98 ns 1.08 0.05 False
Await_FromResult - Duration of single invocation 244.46 ns 258.22 ns 1.06 0.11 False
CreateAndAwait_FromResult - Duration of single invocation 246.63 ns 263.77 ns 1.07 0.12 False
CreateAndAwait_FromCompletedTask - Duration of single invocation 285.93 ns 303.54 ns 1.06 0.08 False
CreateAndAwait_FromCompletedValueTaskSource - Duration of single invocation 351.43 ns 378.30 ns 1.08 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.ValueTaskPerfTest.Await_FromCompletedValueTaskSource

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 595.081229867719 > 594.2430630036507.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.311394562165562 (T) = (0 -602.4053790046316) / Math.Sqrt((68.59264877880152 / (19)) + (59.14862817343829 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06473890695690176 = (565.7775582995725 - 602.4053790046316) / 565.7775582995725 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource_ConfigureAwait

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 450.9786584424671 > 432.3214511336641.
IsChangePoint: Marked as a change because one of 10/3/2022 11:31:18 PM, 11/8/2022 6:56:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.202301400686167 (T) = (0 -443.42573700923776) / Math.Sqrt((124.77035627456547 / (19)) + (40.48379764140281 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05525371058046253 = (420.2077022456741 - 443.42573700923776) / 420.2077022456741 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tasks.ValueTaskPerfTest.Await_FromResult

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 258.2197541050974 > 255.0721417673612.
IsChangePoint: Marked as a change because one of 10/21/2022 3:39:11 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.523990015020129 (T) = (0 -261.28951120096764) / Math.Sqrt((51.69898614240606 / (18)) + (21.310389435704078 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.0660498695956049 = (245.1006455261658 - 261.28951120096764) / 245.1006455261658 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromResult

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 263.7651081994557 > 260.667381061779.
IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.064553033755805 (T) = (0 -268.38210228471974) / Math.Sqrt((22.68424658370055 / (19)) + (35.13853789853432 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06896443116681862 = (251.06738302954537 - 268.38210228471974) / 251.06738302954537 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedTask

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 303.54131541015846 > 300.891896100967.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.30694671564601 (T) = (0 -305.8409191836081) / Math.Sqrt((20.455202996812815 / (19)) + (31.28669174706203 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05793263164152107 = (289.0930008549371 - 305.8409191836081) / 289.0930008549371 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 378.3026742041035 > 371.58166617292426.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.64275199743101 (T) = (0 -383.3910802242919) / Math.Sqrt((69.95539594220982 / (19)) + (41.75001762563019 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07419328235332483 = (356.91070361598713 - 383.3910802242919) / 356.91070361598713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ILogB - Duration of single invocation 114.74 μs 123.18 μs 1.07 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.ILogB

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.17697335489314 > 120.8081067085357.
IsChangePoint: Marked as a change because one of 9/15/2022 1:09:14 PM, 9/16/2022 12:08:08 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.779487628163587 (T) = (0 -123642.78105835036) / Math.Sqrt((3804720.7790189367 / (19)) + (3305189.9924216503 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06709325010571002 = (115868.76877545788 - 123642.78105835036) / 115868.76877545788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 103.59 μs 116.57 μs 1.13 0.04 False
List - Duration of single invocation 8.96 μs 10.41 μs 1.16 0.14 False
ImmutableList - Duration of single invocation 99.64 μs 114.57 μs 1.15 0.05 False
IList - Duration of single invocation 16.65 μs 18.96 μs 1.14 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 116.56916148897056 > 105.80252525456862.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.20965436959416 (T) = (0 -115418.97119174882) / Math.Sqrt((9090745.0866447 / (19)) + (3739763.3351196432 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13205210352823937 = (101955.5291068541 - 115418.97119174882) / 101955.5291068541 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;String&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.414756350729544 > 9.360844742627014.
IsChangePoint: Marked as a change because one of 8/2/2022 2:18:01 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.224077409263263 (T) = (0 -10384.6902534298) / Math.Sqrt((62525.95918733211 / (19)) + (64668.67454837506 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.15118472791496243 = (9020.872151629963 - 10384.6902534298) / 9020.872151629963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;String&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.56815339111924 > 104.87646202946651.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.11928290985501 (T) = (0 -115681.11532531957) / Math.Sqrt((8218469.177590342 / (19)) + (1410112.4679740672 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.1377775912034979 = (101672.87193884393 - 115681.11532531957) / 101672.87193884393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;String&gt;.IList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.95878573309032 > 17.555145084250913.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.765810166646272 (T) = (0 -19146.99686069358) / Math.Sqrt((82574.6795796928 / (19)) + (219306.52167090477 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.135958371102859 = (16855.368425257067 - 19146.99686069358) / 16855.368425257067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsBenchmark - Duration of single invocation 36.07 ns 39.35 ns 1.09 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.EqualsBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.34755799809303 > 37.948967004831914.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.170823526411706 (T) = (0 -40.18712280334072) / Math.Sqrt((1.4178291325850294 / (19)) + (1.6755616450923478 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09132247042634639 = (36.82424204794467 - 40.18712280334072) / 36.82424204794467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 123.14 ns 137.53 ns 1.12 0.24 False
ToString - Duration of single invocation 18.31 ns 23.19 ns 1.27 0.35 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.Parse(value: "0")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 137.53200771942156 > 133.8344599851849.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.32638362341148 (T) = (0 -138.11682987574335) / Math.Sqrt((15.424171817565414 / (19)) + (36.538378346124674 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07139932572579709 = (128.91256001321355 - 138.11682987574335) / 128.91256001321355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int16.ToString(value: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.193319244180195 > 19.337266036764536.
IsChangePoint: Marked as a change because one of 10/15/2022 1:58:20 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.745843231801846 (T) = (0 -22.760795956769307) / Math.Sqrt((0.5377016958942443 / (19)) + (0.7098020965366552 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.2203909153545186 = (18.650414117641468 - 22.760795956769307) / 18.650414117641468 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AndArray - Duration of single invocation 338.78 μs 365.15 μs 1.08 0.05 False
AndOr - Duration of single invocation 404.24 μs 436.64 μs 1.08 0.09 False
Single - Duration of single invocation 295.27 μs 314.63 μs 1.07 0.11 False
And - Duration of single invocation 341.09 μs 370.21 μs 1.09 0.08 False
OrOr - Duration of single invocation 396.86 μs 432.65 μs 1.09 0.02 False
OrArray - Duration of single invocation 346.82 μs 370.64 μs 1.07 0.05 False
AndAndAnd - Duration of single invocation 418.78 μs 456.11 μs 1.09 0.06 False
Or - Duration of single invocation 350.27 μs 372.47 μs 1.06 0.06 False
AndAnd - Duration of single invocation 380.54 μs 422.46 μs 1.11 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'

Payloads

Baseline
Compare

Histogram

IfStatements.IfStatements.AndArray

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 365.14633236434105 > 355.69857886628256.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.886212475556244 (T) = (0 -366308.48355798225) / Math.Sqrt((56030460.771190435 / (19)) + (12264679.885757506 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05785815824082291 = (346273.72365983267 - 366308.48355798225) / 346273.72365983267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.AndOr

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 436.64476953125 > 429.16874698601976.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.455061470899585 (T) = (0 -439511.3968647141) / Math.Sqrt((85124321.42914341 / (19)) + (47246009.16137363 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06920642432427493 = (411063.18374628155 - 439511.3968647141) / 411063.18374628155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.Single

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 314.6333623430141 > 310.26576262902995.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.297674726151788 (T) = (0 -319525.8490751274) / Math.Sqrt((54228439.18131986 / (19)) + (61795435.22152971 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0700477725401643 = (298608.9567913511 - 319525.8490751274) / 298608.9567913511 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.And

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 370.21001211240304 > 358.4319785427989.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.305478547426752 (T) = (0 -372128.0531646679) / Math.Sqrt((94966216.57290322 / (19)) + (55187336.826557495 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07068014984168894 = (347562.2978717695 - 372128.0531646679) / 347562.2978717695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.OrOr

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 432.6492387152778 > 417.01481632582136.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.933087261614011 (T) = (0 -425068.92556058674) / Math.Sqrt((39423323.956185885 / (19)) + (26779221.290835787 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.060523443843726314 = (400810.4941272971 - 425068.92556058674) / 400810.4941272971 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.OrArray

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 370.6389014590593 > 364.0683590579229.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.977518518286967 (T) = (0 -375405.5464597506) / Math.Sqrt((42572032.806193575 / (19)) + (82549507.17896296 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06444351157552736 = (352677.7535654261 - 375405.5464597506) / 352677.7535654261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.AndAndAnd

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 456.10810529761915 > 439.9874075579286.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.53088622482979 (T) = (0 -456780.53841159487) / Math.Sqrt((32531094.499892395 / (19)) + (22279076.408920784 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0847311430080811 = (421100.234243198 - 456780.53841159487) / 421100.234243198 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.Or

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 372.47274585459184 > 367.4745267369003.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.318161769338527 (T) = (0 -375379.4279162749) / Math.Sqrt((70164599.3346709 / (19)) + (13230327.848885523 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.059433768601222126 = (354320.8070589358 - 375379.4279162749) / 354320.8070589358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### IfStatements.IfStatements.AndAnd

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 422.46011045040495 > 399.40739071875004.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.412373497115478 (T) = (0 -417819.23441909376) / Math.Sqrt((110630865.95464817 / (19)) + (29319799.442964777 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08443919099427041 = (385285.9965675118 - 417819.23441909376) / 385285.9965675118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 6.88 μs 7.28 μs 1.06 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: da)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.2844085914863 > 7.2510242601942.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.128556064631738 (T) = (0 -7246.250884500645) / Math.Sqrt((5801.368165389788 / (19)) + (4606.692427208257 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05069228333919007 = (6896.64424056817 - 7246.250884500645) / 6896.64424056817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Bag - Duration of single invocation 60.13 ns 68.94 ns 1.15 0.25 False
Queue - Duration of single invocation 52.37 ns 58.57 ns 1.12 0.17 False
Dictionary - Duration of single invocation 145.15 ns 157.24 ns 1.08 0.10 False
Bag - Duration of single invocation 64.47 ns 78.14 ns 1.21 0.21 False
Queue - Duration of single invocation 65.32 ns 71.00 ns 1.09 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<Int32>.Bag(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.93705833448801 > 62.06258776658271.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/11/2022 10:51:47 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.597512261697501 (T) = (0 -70.8320723031328) / Math.Sqrt((11.43336750729122 / (19)) + (5.938749701022558 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1733087812342711 = (60.3695066771941 - 70.8320723031328) / 60.3695066771941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.571203190315835 > 55.00982656106956.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.86451190591448 (T) = (0 -58.535995022077415) / Math.Sqrt((1.3724673256365298 / (19)) + (7.265652194266386 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10749327377270695 = (52.85449258095528 - 58.535995022077415) / 52.85449258095528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;Int32&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.24466657264594 > 152.5491886978342.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.421451035910454 (T) = (0 -161.10552039835247) / Math.Sqrt((5.7512438372856405 / (19)) + (15.805073665575867 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11197455988715783 = (144.88237969643956 - 161.10552039835247) / 144.88237969643956 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;Int32&gt;.Bag(Size: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.13594227128598 > 68.06413822085543.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.62543445838472 (T) = (0 -76.11668225103641) / Math.Sqrt((3.141737993119476 / (19)) + (15.96730708767011 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.18137812041280857 = (64.43041472990797 - 76.11668225103641) / 64.43041472990797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;Int32&gt;.Queue(Size: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.00403618145938 > 68.95059505087028.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.472088214829267 (T) = (0 -71.85553049187438) / Math.Sqrt((0.9151623958891093 / (18)) + (2.466650291196575 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.09230864576762643 = (65.78317471924565 - 71.85553049187438) / 65.78317471924565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Threading.Tests.Perf_SemaphoreSlim

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReleaseWait - Duration of single invocation 510.14 ns 556.57 ns 1.09 0.11 False
ReleaseWaitAsync - Duration of single invocation 518.40 ns 576.85 ns 1.11 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_SemaphoreSlim*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWait

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 556.5691435997518 > 543.5875379733541.
IsChangePoint: Marked as a change because one of 10/7/2022 11:47:58 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.798720937224445 (T) = (0 -564.260892349697) / Math.Sqrt((113.54371791851047 / (19)) + (181.7193382188669 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07246680545197616 = (526.1336663113756 - 564.260892349697) / 526.1336663113756 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWaitAsync

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 576.8523004791753 > 550.3178290478699.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.05818579303633 (T) = (0 -583.4397119433112) / Math.Sqrt((129.85292112095382 / (19)) + (175.51200924579948 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08815726290103848 = (536.1722352409383 - 583.4397119433112) / 536.1722352409383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyByScalarOperatorBenchmark - Duration of single invocation 16.98 ns 19.27 ns 1.13 0.28 False
TransformByMatrix4x4Benchmark - Duration of single invocation 24.98 ns 28.71 ns 1.15 0.25 False
TransformNormalByMatrix3x2Benchmark - Duration of single invocation 21.43 ns 24.03 ns 1.12 0.26 False
DotBenchmark - Duration of single invocation 10.85 ns 12.55 ns 1.16 0.39 False
TransformNormalByMatrix4x4Benchmark - Duration of single invocation 21.08 ns 24.41 ns 1.16 0.27 False
LengthBenchmark - Duration of single invocation 12.82 ns 15.73 ns 1.23 0.28 False
TransformByMatrix3x2Benchmark - Duration of single invocation 25.15 ns 28.39 ns 1.13 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector2.MultiplyByScalarOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.27063966121164 > 17.92664228061314.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -3.2318688667810713 (T) = (0 -18.792439465787933) / Math.Sqrt((1.08887662059649 / (19)) + (2.0378051744748578 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0661301489891359 = (17.626778009801345 - 18.792439465787933) / 17.626778009801345 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.70734406289442 > 26.16651588090477.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.388477992350555 (T) = (0 -28.2061693677698) / Math.Sqrt((0.18432202898094002 / (19)) + (0.6786359299971658 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12811998538561153 = (25.002809748227648 - 28.2061693677698) / 25.002809748227648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.02507702842308 > 22.314244141747416.
IsChangePoint: Marked as a change because one of 9/23/2022 2:49:56 PM, 9/24/2022 2:54:54 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.311451627074756 (T) = (0 -23.999617382245905) / Math.Sqrt((0.23017523542753007 / (19)) + (0.29351008841265014 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12161849346572468 = (21.397308908565446 - 23.999617382245905) / 21.397308908565446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector2.DotBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.55086982941056 > 11.471171317034193.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.807895329555963 (T) = (0 -12.31358757250771) / Math.Sqrt((0.12047030742547252 / (19)) + (0.507217036283725 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14228286139532684 = (10.779805938317553 - 12.31358757250771) / 10.779805938317553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.40801762828583 > 22.38245030260188.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.805614662404762 (T) = (0 -24.25512896868688) / Math.Sqrt((0.3233219700499315 / (19)) + (0.6705097865720935 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13019871006162623 = (21.460942003167144 - 24.25512896868688) / 21.460942003167144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector2.LengthBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.73411574641506 > 13.481880131147241.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.087661086912174 (T) = (0 -14.820058732073571) / Math.Sqrt((0.20641110754416456 / (19)) + (0.951308584780058 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14922258014481024 = (12.89572532607751 - 14.820058732073571) / 12.89572532607751 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.389044212220895 > 26.087891866026357.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.01209882351651 (T) = (0 -28.900590740196982) / Math.Sqrt((0.8198364693414489 / (19)) + (3.5724740804570234 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.1502459961375896 = (25.12557386614886 - 28.900590740196982) / 25.12557386614886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DelegateInvoke - Duration of single invocation 3.21 ms 3.79 ms 1.18 0.12 False
MulticastDelegateInvoke - Duration of single invocation 159.10 ms 170.40 ms 1.07 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.DelegatePerf.DelegateInvoke

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7928174229166665 > 3.360033720901442.
IsChangePoint: Marked as a change because one of 7/23/2022 5:59:35 PM, 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.380491815847726 (T) = (0 -3600308.110487331) / Math.Sqrt((6856685115.03236 / (19)) + (7176122096.643752 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11015929671161666 = (3243055.4075903706 - 3600308.110487331) / 3243055.4075903706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.DelegatePerf.MulticastDelegateInvoke(length: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 170.39798742857144 > 167.32652593125.
IsChangePoint: Marked as a change because one of 10/27/2022 9:29:16 PM, 10/28/2022 10:12:00 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.09746622860403 (T) = (0 -172981857.17467293) / Math.Sqrt((7431013225874.691 / (19)) + (23026009012863.645 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08346868505326514 = (159655613.08877963 - 172981857.17467293) / 159655613.08877963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 117.38 ns 145.87 ns 1.24 0.20 False
Parse - Duration of single invocation 114.94 ns 125.41 ns 1.09 0.27 False
Parse - Duration of single invocation 150.13 ns 168.63 ns 1.12 0.32 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.TryParse(value: "0")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.8675878013516 > 123.87342064551856.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.877468567171233 (T) = (0 -132.83337253340687) / Math.Sqrt((11.422622311089903 / (19)) + (16.26223679180079 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1178145737035998 = (118.83310135534968 - 132.83337253340687) / 118.83310135534968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Byte.Parse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.40668851654831 > 122.17384024535218.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.95025677817547 (T) = (0 -127.37862425262792) / Math.Sqrt((21.539591036602307 / (19)) + (3.423054462880178 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07520671695266355 = (118.46896252066087 - 127.37862425262792) / 118.46896252066087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Byte.Parse(value: "255")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.62905669332926 > 158.7414424221929.
IsChangePoint: Marked as a change because one of 10/20/2022 10:19:58 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.873281557769742 (T) = (0 -165.6350714516141) / Math.Sqrt((49.70348829586507 / (19)) + (35.375497980505 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08900693580031749 = (152.09735218985352 - 165.6350714516141) / 152.09735218985352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 564.63 μs 606.64 μs 1.07 0.10 False
IList - Duration of single invocation 19.42 μs 20.65 μs 1.06 0.10 False
List - Duration of single invocation 11.08 μs 11.89 μs 1.07 0.11 False
Dictionary - Duration of single invocation 100.70 μs 113.45 μs 1.13 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 606.6403782623627 > 592.7741960379464.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.504910896822869 (T) = (0 -623877.4241945738) / Math.Sqrt((169557510.15994918 / (19)) + (230526738.82847482 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10656528640283337 = (563796.309048012 - 623877.4241945738) / 563796.309048012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSet&lt;Int32&gt;.IList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.649172701719575 > 19.726555846067537.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.950237908969823 (T) = (0 -20845.457690337742) / Math.Sqrt((226575.13550910255 / (19)) + (152139.14179291704 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.09731041907423851 = (18996.86481417383 - 20845.457690337742) / 18996.86481417383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSet&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.89462520340707 > 11.634285774561588.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.373153101738112 (T) = (0 -11955.942028119145) / Math.Sqrt((13547.22938533692 / (19)) + (88885.918824316 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0749689910628122 = (11122.127361365478 - 11955.942028119145) / 11122.127361365478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSet&lt;Int32&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.44513903061224 > 104.61296943821507.
IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.709866736154137 (T) = (0 -113032.9368779297) / Math.Sqrt((3040548.0869816532 / (19)) + (7579168.767694219 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.136679912580059 = (99441.30764250532 - 113032.9368779297) / 99441.30764250532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetPinnableReference - Duration of single invocation 18.41 ns 22.39 ns 1.22 0.11 False
IndexOfAnyTwoValues - Duration of single invocation 3.84 μs 4.31 μs 1.12 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.GetPinnableReference(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.38509751199092 > 19.393064594610596.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -26.01624097497119 (T) = (0 -22.66704584654413) / Math.Sqrt((0.06964615751880873 / (19)) + (0.5618793640801825 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.21886410970317519 = (18.59686052456179 - 22.66704584654413) / 18.59686052456179 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.3110929303539605 > 4.030696523922468.
IsChangePoint: Marked as a change because one of 11/16/2022 6:51:59 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.019098053623598 (T) = (0 -4089.2437588633697) / Math.Sqrt((3502.528248249395 / (23)) + (6408.787725456535 / (24))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (24) - 2, .025) and -0.05282158976061911 = (3884.0804544986054 - 4089.2437588633697) / 3884.0804544986054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same - Duration of single invocation 2.26 μs 2.42 μs 1.07 0.05 False
Compare_DifferentFirstChar - Duration of single invocation 249.77 ns 278.23 ns 1.11 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4158865434304384 > 2.3774220469855654.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.325034023311044 (T) = (0 -2428.725626662423) / Math.Sqrt((562.830909985856 / (19)) + (8413.65566421095 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07499048852613406 = (2259.299642727377 - 2428.725626662423) / 2259.299642727377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (pl-PL, None))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 278.2298634155285 > 261.11125474000215.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 11/17/2022 7:01:39 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -4.844199998653929 (T) = (0 -268.5086458725094) / Math.Sqrt((48.57832241197929 / (27)) + (114.01702907527452 / (20))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (20) - 2, .025) and -0.05197584982225045 = (255.242214845406 - 268.5086458725094) / 255.242214845406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Floor - Duration of single invocation 62.04 ns 67.17 ns 1.08 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Floor

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.16784432750373 > 65.29246080067927.
IsChangePoint: Marked as a change because one of 10/2/2022 4:42:49 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.60498680612237 (T) = (0 -66.94807290539111) / Math.Sqrt((2.590222766343489 / (19)) + (2.4881694440330944 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06495596074572503 = (62.86463982840321 - 66.94807290539111) / 62.86463982840321 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchstone.BenchF.Bisect

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 2.81 secs 3.05 secs 1.09 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Bisect*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Bisect.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.052698743857143 > 2.9485390973250003.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.851420933911976 (T) = (0 -3058283297.494611) / Math.Sqrt((3242886742263243.5 / (19)) + (2256979293793943 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07734925070593128 = (2838711119.4356666 - 3058283297.494611) / 2838711119.4356666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetNames - Duration of single invocation 183.87 ns 201.38 ns 1.10 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.GetNames

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.37923819468614 > 193.01556433967318.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.050685903878927 (T) = (0 -199.10966267788388) / Math.Sqrt((20.92008392270911 / (19)) + (12.245974630665208 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.06716547522573553 = (186.5780586986911 - 199.10966267788388) / 186.5780586986911 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 36.33 ns 38.38 ns 1.06 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.DotBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.37579409839841 > 38.018589191107736.
IsChangePoint: Marked as a change because one of 9/20/2022 10:35:57 PM, 9/21/2022 2:58:31 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.750855943728869 (T) = (0 -38.428927305852774) / Math.Sqrt((0.2779704162901401 / (19)) + (0.21150267462397015 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05629551397236007 = (36.3808487279615 - 38.428927305852774) / 36.3808487279615 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in PerfLabTests.EnumPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ObjectGetTypeNoBoxing - Duration of single invocation 3.56 ns 4.79 ns 1.34 0.65 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.EnumPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.EnumPerf.ObjectGetTypeNoBoxing

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.79007010579725 > 3.4067062810406874.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.349810758331236 (T) = (0 -4.915232965238862) / Math.Sqrt((0.09568930025430544 / (19)) + (0.5639713846564178 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.5017676291110483 = (3.2729650512898387 - 4.915232965238862) / 3.2729650512898387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Combine - Duration of single invocation 73.14 ns 84.04 ns 1.15 0.13 False
IsPathRooted - Duration of single invocation 37.05 ns 40.68 ns 1.10 0.23 False
GetPathRoot - Duration of single invocation 52.88 ns 57.26 ns 1.08 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.Combine

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.03591496417387 > 76.01628430820925.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.683143732055136 (T) = (0 -79.40523709453736) / Math.Sqrt((1.9566273826821912 / (19)) + (2.8099833052139824 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0909683211873682 = (72.78418222823898 - 79.40523709453736) / 72.78418222823898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.IsPathRooted

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.678802713342115 > 38.98749821737649.
IsChangePoint: Marked as a change because one of 7/24/2022 1:26:55 PM, 10/24/2022 9:01:44 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.0699958839544 (T) = (0 -40.90648337342357) / Math.Sqrt((2.6435289100058426 / (19)) + (0.9969084813466452 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07787302102422687 = (37.95111536844388 - 40.90648337342357) / 37.95111536844388 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetPathRoot

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.26020775516684 > 55.73545743048673.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.458165852984973 (T) = (0 -57.39777932598657) / Math.Sqrt((0.6772032407709772 / (19)) + (1.50990396918701 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07546808674775526 = (53.37004420052948 - 57.39777932598657) / 53.37004420052948 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_ToLower - Duration of single invocation 225.10 ns 236.79 ns 1.05 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLower(c: '你', cultureName: zh-Hans)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.7914588609574 > 236.0962237852751.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.588458901487272 (T) = (0 -238.40543131080034) / Math.Sqrt((14.591562495414077 / (19)) + (29.958283877907657 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05134611667309509 = (226.76207913833002 - 238.40543131080034) / 226.76207913833002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in XmlDocumentTests.XmlNodeTests.Perf_XmlNode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetValue - Duration of single invocation 12.67 ns 13.77 ns 1.09 0.38 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlNodeTests.Perf_XmlNode*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlNodeTests.Perf_XmlNode.GetValue

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.765038114601776 > 13.1826042048896.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.2451600366728 (T) = (0 -14.742956528478825) / Math.Sqrt((1.1456805410599233 / (19)) + (0.2652318817876721 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12598889216718753 = (13.09334100099611 - 14.742956528478825) / 13.09334100099611 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CallBlockCopy - Duration of single invocation 75.40 ns 81.71 ns 1.08 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.71498245298297 > 79.10202325486848.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.937665309021214 (T) = (0 -83.29646709003696) / Math.Sqrt((13.355718428461396 / (19)) + (3.5420068035009713 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08207987437462802 = (76.97811322678643 - 83.29646709003696) / 76.97811322678643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqOrderByExtension - Duration of single invocation 679.94 μs 743.12 μs 1.09 0.03 False
LinqQuery - Duration of single invocation 676.12 μs 738.35 μs 1.09 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 743.1165051285715 > 703.6318078085002.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.800931596062515 (T) = (0 -740270.810578718) / Math.Sqrt((43070347.38171247 / (19)) + (163860359.13085195 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10085185995672846 = (672452.7045880778 - 740270.810578718) / 672452.7045880778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqQuery(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 738.3455916933335 > 710.3896370819999.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.733460493839825 (T) = (0 -738279.3884359392) / Math.Sqrt((103325003.69422238 / (19)) + (97097411.16637918 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09147543376613453 = (676404.9520459725 - 738279.3884359392) / 676404.9520459725 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EmptyAsyncMethodInvocation - Duration of single invocation 192.89 ns 207.90 ns 1.08 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.Tests.Perf_AsyncMethods.EmptyAsyncMethodInvocation

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 207.89562493589744 > 201.31257516346156.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.96014640022651 (T) = (0 -209.70780610882463) / Math.Sqrt((14.203790104599312 / (19)) + (23.213228690746494 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0841275151385667 = (193.43463124079184 - 209.70780610882463) / 193.43463124079184 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in SIMD.ConsoleMandel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScalarDoubleSinglethreadRaw - Duration of single invocation 3.43 secs 3.67 secs 1.07 0.11 False
ScalarFloatSinglethreadRaw - Duration of single invocation 3.42 secs 3.67 secs 1.07 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SIMD.ConsoleMandel*'

Payloads

Baseline
Compare

Histogram

SIMD.ConsoleMandel.ScalarDoubleSinglethreadRaw

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.671170300785714 > 3.5964204897088465.
IsChangePoint: Marked as a change because one of 10/6/2022 11:18:47 AM, 10/6/2022 8:38:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.766082572075523 (T) = (0 -3681180862.9731684) / Math.Sqrt((3776947217778281 / (19)) + (31115143428016432 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07126734575326875 = (3436285888.453663 - 3681180862.9731684) / 3436285888.453663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### SIMD.ConsoleMandel.ScalarFloatSinglethreadRaw

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.665066219285714 > 3.59103922638.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.019833191652403 (T) = (0 -3622105115.076884) / Math.Sqrt((724527910731096 / (19)) + (1975811800987607.8 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.057917208738691064 = (3423807728.201494 - 3622105115.076884) / 3423807728.201494 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EmptyNameSpaceToString - Duration of single invocation 21.93 ns 24.43 ns 1.11 0.40 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XName*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XName.EmptyNameSpaceToString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.42863016142753 > 22.966526453194035.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.919742967738932 (T) = (0 -24.682390121521646) / Math.Sqrt((0.3977534675220229 / (19)) + (0.9772563965425832 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11674489901926897 = (22.102084498615437 - 24.682390121521646) / 22.102084498615437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PathAndQuery - Duration of single invocation 31.70 ns 42.85 ns 1.35 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.PathAndQuery

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.84610438951677 > 33.337486644348786.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.035620371163786 (T) = (0 -36.75723103835565) / Math.Sqrt((2.3841724353002127 / (19)) + (4.521257131567938 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1516504476356328 = (31.917003213795613 - 36.75723103835565) / 31.917003213795613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 18.56 μs 19.66 μs 1.06 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<Location>.JsonNet_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.65655830255403 > 19.390943886662377.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.36611328909216 (T) = (0 -19631.15763661757) / Math.Sqrt((39007.13925165999 / (19)) + (45436.766751641735 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.056672179700007826 = (18578.285691396653 - 19631.15763661757) / 18578.285691396653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IterateForNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 17.71 μs 19.46 μs 1.10 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForNonGeneric&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForNonGeneric<String>.ArrayList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.457719333748443 > 17.60052204767477.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.298018794682132 (T) = (0 -19641.5594368306) / Math.Sqrt((176564.1148706559 / (19)) + (325455.88178126316 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.15571108391512523 = (16995.21594124736 - 19641.5594368306) / 16995.21594124736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
benchMonteCarlo - Duration of single invocation 4.17 secs 4.43 secs 1.06 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'

Payloads

Baseline
Compare

Histogram

SciMark2.kernel.benchMonteCarlo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.4253215332142855 > 4.3656443359425.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.155801989003095 (T) = (0 -4436273252.466941) / Math.Sqrt((3562231869014482 / (19)) + (5151178582378229 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05575481924437497 = (4201991950.7846255 - 4436273252.466941) / 4201991950.7846255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 97.61 μs 106.08 μs 1.09 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.07793330150159 > 104.32846860090882.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -3.704847718932131 (T) = (0 -105841.84127699597) / Math.Sqrt((38923836.46897931 / (19)) + (4354156.080598739 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.054816106840068095 = (100341.51032644762 - 105841.84127699597) / 100341.51032644762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForEach_Array - Duration of single invocation 137.49 ns 152.84 ns 1.11 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 152.83861020661735 > 144.19553807759868.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.201920695045683 (T) = (0 -155.56641400411354) / Math.Sqrt((7.399557617763431 / (19)) + (16.455017153864823 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11476517865382815 = (139.5508372372852 - 155.56641400411354) / 139.5508372372852 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NestedScopes_TwoMessages - Duration of single invocation 806.95 ns 883.94 ns 1.10 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: True)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 883.9391255360737 > 862.4999456756461.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.554745866621682 (T) = (0 -879.7015857124268) / Math.Sqrt((249.10432024411608 / (19)) + (102.29891287290575 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.067336394362679 = (824.2027446629968 - 879.7015857124268) / 824.2027446629968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDocumentElement - Duration of single invocation 68.27 ns 74.45 ns 1.09 0.32 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.45240337999012 > 70.52064216112511.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.466527873169033 (T) = (0 -75.65865731354374) / Math.Sqrt((5.773447468881423 / (19)) + (22.948278960343263 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11682070457717017 = (67.74467647623726 - 75.65865731354374) / 67.74467647623726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Order00LinqMethodX - Duration of single invocation 1.45 secs 1.58 secs 1.09 0.02 False
Count00LinqMethodX - Duration of single invocation 4.65 secs 4.95 secs 1.06 0.08 False
Where00LinqMethodX - Duration of single invocation 3.85 secs 4.20 secs 1.09 0.10 False
Order00ManualX - Duration of single invocation 1.24 secs 1.39 secs 1.12 0.05 False
Where01LinqMethodNestedX - Duration of single invocation 3.68 secs 4.10 secs 1.11 0.12 False
Where01LinqQueryX - Duration of single invocation 3.04 secs 3.36 secs 1.11 0.19 False
Order00LinqQueryX - Duration of single invocation 1.47 secs 1.57 secs 1.07 0.01 False
Where00LinqQueryX - Duration of single invocation 3.82 secs 4.10 secs 1.07 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Order00LinqMethodX

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5801054847333333 > 1.5195920863576922.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.72523135406616 (T) = (0 -1570916281.5272696) / Math.Sqrt((447431606546897.56 / (19)) + (159025962628538.03 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07680450632176146 = (1458868598.9932716 - 1570916281.5272696) / 1458868598.9932716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Count00LinqMethodX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.949179668571428 > 4.904623499503846.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.017285433051637 (T) = (0 -5010899189.138842) / Math.Sqrt((9712322818155356 / (19)) + (10686768763186252 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06352896643069215 = (4711577537.898109 - 5010899189.138842) / 4711577537.898109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where00LinqMethodX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.204505722785714 > 4.0103060741048075.
IsChangePoint: Marked as a change because one of 11/15/2022 9:22:46 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.484575403210897 (T) = (0 -4118062450.8224974) / Math.Sqrt((1843285093348412.2 / (16)) + (4827315555583247 / (30))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (30) - 2, .025) and -0.08059842179797323 = (3810909184.9038467 - 4118062450.8224974) / 3810909184.9038467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Order00ManualX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3911070467692308 > 1.3005925063600001.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.631576566607743 (T) = (0 -1338106677.884262) / Math.Sqrt((761573959872903.9 / (19)) + (554978894824515.06 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0788210831481739 = (1240341608.8045397 - 1338106677.884262) / 1240341608.8045397 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodNestedX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.103620610571428 > 3.8283828138000002.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.640651720454857 (T) = (0 -4038411186.4481225) / Math.Sqrt((5086404559818934 / (19)) + (8594205063324183 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1023433557709387 = (3663478502.6883073 - 4038411186.4481225) / 3663478502.6883073 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqQueryX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.364811423 > 3.2038235667375.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.539527472809484 (T) = (0 -3336849501.6314435) / Math.Sqrt((5768373607558355 / (19)) + (14227992405155318 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07864762372031239 = (3093549207.5924425 - 3336849501.6314435) / 3093549207.5924425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Order00LinqQueryX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5730908174615386 > 1.5299575566288461.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.500724953286316 (T) = (0 -1580336605.6983974) / Math.Sqrt((427268445789875.5 / (19)) + (353228879839054.4 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07887614352612503 = (1464798916.1511471 - 1580336605.6983974) / 1464798916.1511471 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where00LinqQueryX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.0979609885 > 3.9953586688875005.
IsChangePoint: Marked as a change because one of 7/30/2022 7:04:11 AM, 9/28/2022 9:30:44 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.38872707772852 (T) = (0 -4110473952.5347) / Math.Sqrt((7951783886976616 / (19)) + (2078655921632993.8 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07170059196871087 = (3835468584.545401 - 4110473952.5347) / 3835468584.545401 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 536.39 μs 607.46 μs 1.13 0.05 False
ImmutableSortedSet - Duration of single invocation 629.40 μs 675.46 μs 1.07 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 607.4560889423076 > 563.0496323706897.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.03178666987741 (T) = (0 -591117.4262710467) / Math.Sqrt((134089574.89663135 / (19)) + (114032240.69879442 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09270097721482297 = (540969.0652768897 - 591117.4262710467) / 540969.0652768897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrueComparer&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 675.4563836805556 > 637.9773837439903.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.862996456456777 (T) = (0 -673949.2509925804) / Math.Sqrt((106806319.67827289 / (19)) + (92908767.2988344 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09663577704599695 = (614560.7001879828 - 673949.2509925804) / 614560.7001879828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TransformByQuaternionBenchmark - Duration of single invocation 88.78 ns 96.55 ns 1.09 0.20 False
TransformNormalByMatrix4x4Benchmark - Duration of single invocation 37.11 ns 45.55 ns 1.23 0.21 False
LengthBenchmark - Duration of single invocation 16.34 ns 18.74 ns 1.15 0.23 False
CrossBenchmark - Duration of single invocation 27.84 ns 32.36 ns 1.16 0.20 False
TransformByMatrix4x4Benchmark - Duration of single invocation 43.09 ns 51.75 ns 1.20 0.21 False
LengthSquaredBenchmark - Duration of single invocation 14.50 ns 16.39 ns 1.13 0.30 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.5542705276826 > 93.32791365799383.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.423927119658554 (T) = (0 -96.6068238113918) / Math.Sqrt((1.1391268817952822 / (19)) + (2.7201125771541417 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09145941280235438 = (88.51160444285411 - 96.6068238113918) / 88.51160444285411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.5481730168247 > 38.8143633359896.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.033194159310096 (T) = (0 -44.69565488581977) / Math.Sqrt((6.593586997376179 / (19)) + (4.3329140207866494 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.16708356148760783 = (38.29687638548266 - 44.69565488581977) / 38.29687638548266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.LengthBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.74455418556881 > 17.068815406094068.
IsChangePoint: Marked as a change because one of 11/16/2022 11:33:39 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.37312352245221 (T) = (0 -18.69077555043155) / Math.Sqrt((0.7412024269016083 / (24)) + (0.7150499913416064 / (23))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (23) - 2, .025) and -0.1255200637548902 = (16.606345948269055 - 18.69077555043155) / 16.606345948269055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.CrossBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.36187828971306 > 29.439090575759355.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.028060208145902 (T) = (0 -32.22694528813641) / Math.Sqrt((0.43319378162003797 / (19)) + (1.3689980651800264 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.1447230265953694 = (28.15261381085839 - 32.22694528813641) / 28.15261381085839 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.74981982845539 > 45.09999950438432.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.49893380828685 (T) = (0 -50.93176970228294) / Math.Sqrt((0.5891944681537601 / (19)) + (3.3343327866189116 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1847327592661859 = (42.99009148175296 - 50.93176970228294) / 42.99009148175296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.LengthSquaredBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.386089409611476 > 15.270537168466502.
IsChangePoint: Marked as a change because one of 11/15/2022 9:22:46 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.143791463700499 (T) = (0 -16.392394398661263) / Math.Sqrt((0.44263551988756894 / (16)) + (1.2347843605381714 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.10787625672050634 = (14.796232249967531 - 16.392394398661263) / 14.796232249967531 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionaryDeepCopy - Duration of single invocation 98.43 μs 104.79 μs 1.06 0.04 False
ImmutableSortedSet - Duration of single invocation 324.32 μs 347.20 μs 1.07 0.01 False
SortedDictionary - Duration of single invocation 710.89 μs 786.07 μs 1.11 0.05 False
ImmutableSortedDictionary - Duration of single invocation 933.92 μs 1.00 ms 1.07 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.SortedDictionaryDeepCopy(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 104.78941059722221 > 103.31668610384449.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.67333850818354 (T) = (0 -105127.3665758908) / Math.Sqrt((3501945.5857860167 / (19)) + (1387213.3475035692 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05163248034201467 = (99965.88023004102 - 105127.3665758908) / 99965.88023004102 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 347.2002358969156 > 342.56316806848406.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.457899558176084 (T) = (0 -346413.68269277405) / Math.Sqrt((15079352.818339506 / (19)) + (9473407.609423839 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06009708905941442 = (326775.4305411161 - 346413.68269277405) / 326775.4305411161 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 786.0749512499998 > 748.2894751207389.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 10/14/2022 1:19:37 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.204521498778938 (T) = (0 -780705.8448998614) / Math.Sqrt((249784433.529969 / (19)) + (203414051.52447546 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08956268888787236 = (716531.3688345328 - 780705.8448998614) / 716531.3688345328 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorFromCollection&lt;Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0030032416294643 > 981.5059212705033.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 9:30:44 PM, 10/11/2022 6:23:40 AM, 10/14/2022 1:19:37 PM, 10/22/2022 10:22:58 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.55639204002243 (T) = (0 -1010931.1673905972) / Math.Sqrt((295023671.1778957 / (19)) + (224915144.07488438 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08072185464395665 = (935422.1560770121 - 1010931.1673905972) / 935422.1560770121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Log2_uint - Duration of single invocation 34.34 μs 38.25 μs 1.11 0.15 False
PopCount_uint - Duration of single invocation 29.11 μs 32.44 μs 1.11 0.13 False
LeadingZeroCount_ulong - Duration of single invocation 42.03 μs 45.13 μs 1.07 0.07 False
Log2_ulong - Duration of single invocation 40.28 μs 43.34 μs 1.08 0.11 False
PopCount_ulong - Duration of single invocation 28.74 μs 30.89 μs 1.07 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BitOperations.Log2_uint

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.24684964846446 > 36.32118086343039.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.673698489843511 (T) = (0 -36819.82863197451) / Math.Sqrt((1532043.102049739 / (19)) + (536918.4994763444 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05117346657954018 = (35027.35733216724 - 36819.82863197451) / 35027.35733216724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.PopCount_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.44062607003504 > 30.490504183309113.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/6/2022 12:41:43 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.085541256767327 (T) = (0 -31219.12243262113) / Math.Sqrt((95626.43038576616 / (19)) + (1846444.679786034 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07411375140901183 = (29065.005816812412 - 31219.12243262113) / 29065.005816812412 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.1273815568933 > 44.084125784651675.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.996137645240522 (T) = (0 -44674.70212232323) / Math.Sqrt((330154.30731324904 / (18)) + (532950.2186016393 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.054246315353621045 = (42375.96230757344 - 44674.70212232323) / 42375.96230757344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.Log2_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.34166492030696 > 42.293274681862115.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -3.772406190160888 (T) = (0 -43190.72250912627) / Math.Sqrt((6067612.124644338 / (19)) + (536707.4745096988 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05353874356151286 = (40995.855893366584 - 43190.72250912627) / 40995.855893366584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.PopCount_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.890027443516704 > 30.146519998592865.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.837776155501245 (T) = (0 -31261.57705391329) / Math.Sqrt((111756.06328106871 / (19)) + (4876266.48943596 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08605281787681371 = (28784.58260900084 - 31261.57705391329) / 28784.58260900084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in GuardedDevirtualization.TwoClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 17.32 ns 18.93 ns 1.09 0.06 False
Call - Duration of single invocation 17.03 ns 18.86 ns 1.11 0.09 False
Call - Duration of single invocation 17.07 ns 18.88 ns 1.11 0.09 False
Call - Duration of single invocation 17.25 ns 18.87 ns 1.09 0.09 False
Call - Duration of single invocation 17.82 ns 19.06 ns 1.07 0.08 False
Call - Duration of single invocation 17.10 ns 20.05 ns 1.17 0.04 False
Call - Duration of single invocation 17.06 ns 19.09 ns 1.12 0.06 False
Call - Duration of single invocation 17.31 ns 18.90 ns 1.09 0.07 False
Call - Duration of single invocation 17.38 ns 18.95 ns 1.09 0.05 False
Call - Duration of single invocation 17.10 ns 18.89 ns 1.10 0.11 False
Call - Duration of single invocation 17.09 ns 18.87 ns 1.10 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassVirtual*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.93152263141809 > 18.055789222980824.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.16241187732954 (T) = (0 -19.039826093172596) / Math.Sqrt((0.10076435632086879 / (19)) + (0.10822917312977508 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10664751438939381 = (17.204959886145907 - 19.039826093172596) / 17.204959886145907 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.20)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.85524959465381 > 17.936112126313155.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -28.121972140866443 (T) = (0 -19.01229312874034) / Math.Sqrt((0.014416701097215826 / (19)) + (0.10910284326369875 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.11417414438184137 = (17.064022912942942 - 19.01229312874034) / 17.064022912942942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.875912001641645 > 17.92815290539617.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.276983466132066 (T) = (0 -19.080149444606253) / Math.Sqrt((0.026484723436660387 / (19)) + (0.19002575410749942 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11806870570658448 = (17.065274564274826 - 19.080149444606253) / 17.065274564274826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.30)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.86864716457416 > 17.94050403829642.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.878868202688468 (T) = (0 -18.989364011713032) / Math.Sqrt((0.20436375222164904 / (19)) + (0.06431096334960835 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10562559486534247 = (17.17522106932212 - 18.989364011713032) / 17.17522106932212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.80)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.06078165454972 > 18.161901577851815.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.8478566933776 (T) = (0 -19.05144524328126) / Math.Sqrt((0.08137446681178896 / (19)) + (0.12029062394994826 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10680024978546099 = (17.21308361375428 - 19.05144524328126) / 17.21308361375428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 1.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.053495226438187 > 17.936227051728668.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -29.541864651286517 (T) = (0 -18.94730084616616) / Math.Sqrt((0.031129951110866513 / (19)) + (0.06330838173354324 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10786432022166413 = (17.102546313952182 - 18.94730084616616) / 17.102546313952182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.70)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.09084790316521 > 17.96179782934563.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.201741552940975 (T) = (0 -18.961707787383183) / Math.Sqrt((0.3474308309012877 / (19)) + (0.02915282443067991 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.08957354499707644 = (17.402870943818723 - 18.961707787383183) / 17.402870943818723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.60)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.89702278475493 > 17.961479623318073.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.14637967093197 (T) = (0 -18.960684949342117) / Math.Sqrt((0.18622200993675814 / (19)) + (0.029080675248915402 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09728608396207557 = (17.279618530182177 - 18.960684949342117) / 17.279618530182177 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.90)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.954456516981793 > 17.954233831035445.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.904905674666416 (T) = (0 -18.996118850924038) / Math.Sqrt((0.05252139415526181 / (19)) + (0.07113305913411545 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10556952164597408 = (17.18220200448596 - 18.996118850924038) / 17.18220200448596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.50)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.885221761844715 > 17.981982933764176.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.468871524624713 (T) = (0 -19.187692549915614) / Math.Sqrt((0.05416591527959017 / (19)) + (0.2926856752063951 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11733118734937693 = (17.17278884467031 - 19.187692549915614) / 17.17278884467031 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.40)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.87062788000871 > 17.947355639889103.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.810200203802967 (T) = (0 -19.042920417243312) / Math.Sqrt((0.029245969131030898 / (19)) + (0.12468699426545272 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11216725038382742 = (17.122353144881117 - 19.042920417243312) / 17.122353144881117 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.IO.Tests.StringReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadToEndAsync - Duration of single invocation 372.25 ns 409.36 ns 1.10 0.03 False
ReadToEndAsync - Duration of single invocation 374.16 ns 402.10 ns 1.07 0.07 False
ReadToEndAsync - Duration of single invocation 370.88 ns 408.79 ns 1.10 0.05 False
ReadToEndAsync - Duration of single invocation 379.30 ns 410.15 ns 1.08 0.04 False
ReadToEndAsync - Duration of single invocation 368.80 ns 410.94 ns 1.11 0.03 False
ReadToEndAsync - Duration of single invocation 374.44 ns 404.78 ns 1.08 0.05 False
ReadToEndAsync - Duration of single invocation 374.94 ns 407.79 ns 1.09 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadToEndTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 0, 1024])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 409.36281215259896 > 390.1685803246242.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -25.311375009172984 (T) = (0 -409.36414556624794) / Math.Sqrt((24.603831752212997 / (19)) + (16.34191612357844 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09259319893032748 = (374.6720608978935 - 409.36414556624794) / 374.6720608978935 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 402.09715511701154 > 394.9602208779899.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.239293892702683 (T) = (0 -408.5646429958779) / Math.Sqrt((34.35430985733248 / (19)) + (19.978083416842352 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.08586462648972534 = (376.2574385691565 - 408.5646429958779) / 376.2574385691565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 408.79008361501417 > 394.63525137650606.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -26.354307016611152 (T) = (0 -410.0131469890378) / Math.Sqrt((11.858007888666107 / (19)) + (36.241318292552656 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09772886142231588 = (373.5104007903992 - 410.0131469890378) / 373.5104007903992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 410.1503856676839 > 395.3389231201009.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.713500766572462 (T) = (0 -407.99989379857453) / Math.Sqrt((20.072206333851856 / (19)) + (24.910900763746213 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09229948987386767 = (373.52383442538087 - 407.99989379857453) / 373.52383442538087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 410.93513420213793 > 390.5990239028496.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.20162657299383 (T) = (0 -411.74542674795896) / Math.Sqrt((29.012400760407324 / (19)) + (45.31687212199267 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10050108706782511 = (374.14358930349937 - 411.74542674795896) / 374.14358930349937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 404.7842418888313 > 393.5499668951278.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.65458889922521 (T) = (0 -408.88827468676755) / Math.Sqrt((39.35317914019184 / (19)) + (28.491935003935414 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08717121434655507 = (376.1029259154273 - 408.88827468676755) / 376.1029259154273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 407.79176588111505 > 392.89564875377647.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.651527902305556 (T) = (0 -408.656204735274) / Math.Sqrt((11.236559949555526 / (19)) + (36.808380863456286 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08684139215311233 = (376.0035343572038 - 408.656204735274) / 376.0035343572038 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Devirtualization.EqualityComparer

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ValueTupleCompareCached - Duration of single invocation 270.48 ns 298.82 ns 1.10 0.09 False
ValueTupleCompare - Duration of single invocation 279.19 ns 300.37 ns 1.08 0.10 False
ValueTupleCompareWrapped - Duration of single invocation 276.83 ns 294.20 ns 1.06 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Devirtualization.EqualityComparer*'

Payloads

Baseline
Compare

Histogram

Devirtualization.EqualityComparer.ValueTupleCompareCached

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 298.8226930538214 > 285.5671359171768.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.58756375318997 (T) = (0 -298.501498645756) / Math.Sqrt((67.41487551967879 / (19)) + (36.77557948075208 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06772864652152402 = (279.56681654858886 - 298.501498645756) / 279.56681654858886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Devirtualization.EqualityComparer.ValueTupleCompare

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 300.3731992851662 > 294.85414333651596.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.31617260908131 (T) = (0 -300.78944346955035) / Math.Sqrt((39.84245899445032 / (19)) + (25.136990150581862 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06309675107573892 = (282.9370357544447 - 300.78944346955035) / 282.9370357544447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Devirtualization.EqualityComparer.ValueTupleCompareWrapped

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 294.20009675418044 > 291.428797684122.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.378430498370314 (T) = (0 -300.69473234129777) / Math.Sqrt((41.66286972949585 / (19)) + (40.63710423305786 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06330806788139033 = (282.7917340460165 - 300.69473234129777) / 282.7917340460165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Singleton - Duration of single invocation 295.17 ns 368.86 ns 1.25 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Singleton

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 368.8589082612072 > 308.55798153819427.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.733868270347977 (T) = (0 -320.0024048964935) / Math.Sqrt((126.82511964601647 / (19)) + (156.29808344363215 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07953170615253413 = (296.4270554284936 - 320.0024048964935) / 296.4270554284936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 636.84 μs 702.08 μs 1.10 0.12 False
ImmutableSortedDictionary - Duration of single invocation 601.97 μs 642.05 μs 1.07 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 702.0772502029222 > 664.5623498365385.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.288218389135888 (T) = (0 -704107.9372172297) / Math.Sqrt((193223237.6790378 / (19)) + (302131575.83663106 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1024193205606326 = (638693.3937797438 - 704107.9372172297) / 638693.3937797438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 642.0476039663462 > 620.1823721538565.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.42498063073396 (T) = (0 -648637.9377644792) / Math.Sqrt((98043636.7216853 / (19)) + (224437547.23723066 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09447500512598354 = (592647.556798079 - 648637.9377644792) / 592647.556798079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EmptyStaticFunction - Duration of single invocation 12.22 ms 14.18 ms 1.16 0.16 False
ForeachOverList100Elements - Duration of single invocation 255.51 ms 273.42 ms 1.07 0.16 False
InterfaceInterfaceMethodLongHierarchy - Duration of single invocation 2.13 ms 2.33 ms 1.09 0.10 False
ClassVirtualMethod - Duration of single invocation 1.97 ms 2.15 ms 1.09 0.23 False
EmptyInstanceFunction - Duration of single invocation 13.01 ms 14.97 ms 1.15 0.14 False
GenericGenericMethod - Duration of single invocation 1.47 ms 1.76 ms 1.19 0.11 False
MeasureEvents - Duration of single invocation 157.29 ms 167.60 ms 1.07 0.13 False
StructWithInterfaceInterfaceMethod - Duration of single invocation 1.54 ms 1.64 ms 1.07 0.15 False
SealedClassInterfaceMethod - Duration of single invocation 1.97 ms 2.15 ms 1.09 0.22 False
InterfaceInterfaceMethodSwitchCallType - Duration of single invocation 3.84 ms 4.43 ms 1.15 0.11 False
InterfaceInterfaceMethod - Duration of single invocation 25.48 ms 28.04 ms 1.10 0.17 False
GenericClassGenericInstanceMethod - Duration of single invocation 1.51 ms 1.83 ms 1.21 0.18 False
StaticDelegate - Duration of single invocation 1.80 ms 2.01 ms 1.12 0.13 False
EmptyStaticFunction5Arg - Duration of single invocation 15.33 ms 16.81 ms 1.10 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.EmptyStaticFunction

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.178289392857144 > 12.825504193632812.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.094275896076653 (T) = (0 -14150544.741834892) / Math.Sqrt((85404582474.72356 / (19)) + (55138237552.90099 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.15103481577879074 = (12293759.100814536 - 14150544.741834892) / 12293759.100814536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.ForeachOverList100Elements

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 273.4185421428571 > 269.480477245.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.109567830893418 (T) = (0 -278203227.75876504) / Math.Sqrt((40129223993179.12 / (19)) + (294076472212904.06 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06978440424184539 = (260055415.51704267 - 278203227.75876504) / 260055415.51704267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3307574717261903 > 2.2159425331219955.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.45465903721206 (T) = (0 -2363407.0921497145) / Math.Sqrt((1344159707.8646877 / (19)) + (3755072464.7776637 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11067813633522901 = (2127895.5755336685 - 2363407.0921497145) / 2127895.5755336685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.ClassVirtualMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.15100825625 > 2.074904810703125.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.306235904972677 (T) = (0 -2202584.451175898) / Math.Sqrt((4672623116.1327505 / (19)) + (7499111692.320683 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10591039636053076 = (1991648.2008166667 - 2202584.451175898) / 1991648.2008166667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.EmptyInstanceFunction

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.973090622916667 > 13.865909670703125.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.86468360814562 (T) = (0 -15108272.8474825) / Math.Sqrt((114283131085.86418 / (19)) + (94569936894.0848 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14603347196758265 = (13183099.11275424 - 15108272.8474825) / 13183099.11275424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericGenericMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7582552263888889 > 1.5441157116477273.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.48866228028803 (T) = (0 -1749490.1385045333) / Math.Sqrt((957515482.4333507 / (19)) + (2551106561.7564883 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.1734939917591663 = (1490838.5989108477 - 1749490.1385045333) / 1490838.5989108477 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.MeasureEvents

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.60329407142856 > 166.11100669750002.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -3.7807835978356152 (T) = (0 -169555388.5472324) / Math.Sqrt((67814650396121.234 / (19)) + (36611535209812.016 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.05206219417092011 = (161164795.66196263 - 169555388.5472324) / 161164795.66196263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6395176179166666 > 1.6312886085238198.
IsChangePoint: Marked as a change because one of 10/16/2022 12:49:12 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.954695303271484 (T) = (0 -1648205.7398583037) / Math.Sqrt((582952955.1186075 / (19)) + (988802354.0487487 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06261251760923828 = (1551088.2024678064 - 1648205.7398583037) / 1551088.2024678064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.SealedClassInterfaceMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1473904352678574 > 2.077766916796875.
IsChangePoint: Marked as a change because one of 7/26/2022 4:40:06 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.655460888632978 (T) = (0 -2167823.755050012) / Math.Sqrt((3094005793.4377995 / (19)) + (2308511070.682178 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08340214141131978 = (2000940.9915195887 - 2167823.755050012) / 2000940.9915195887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodSwitchCallType

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.43033860546875 > 4.055842946601563.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.803264995394576 (T) = (0 -4381802.340003768) / Math.Sqrt((3858578076.9695263 / (19)) + (87162216789.32909 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.130826423703328 = (3874867.3077994264 - 4381802.340003768) / 3874867.3077994264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.042486726495728 > 26.705802221250003.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.662558692082786 (T) = (0 -28435666.741618514) / Math.Sqrt((236148410237.64157 / (19)) + (481968512598.42267 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11220639167194114 = (25566897.43427222 - 28435666.741618514) / 25566897.43427222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8322298377976192 > 1.5948646310227275.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.342726707490064 (T) = (0 -1868656.5989030863) / Math.Sqrt((4356093230.723319 / (19)) + (13417385901.095531 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.21330998274973162 = (1540131.2323072944 - 1868656.5989030863) / 1540131.2323072944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.StaticDelegate

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.008863021763393 > 1.885097217638889.
IsChangePoint: Marked as a change because one of 10/14/2022 3:33:41 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.545579471673506 (T) = (0 -2015281.3467030928) / Math.Sqrt((1042989413.033465 / (19)) + (658845162.8149275 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1154071229409024 = (1806767.506907761 - 2015281.3467030928) / 1806767.506907761 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.809716833333336 > 16.25076723938702.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.1764315651261645 (T) = (0 -16985798.96312772) / Math.Sqrt((573591139765.8883 / (19)) + (1000319404910.8243 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08488253815702909 = (15656809.254191486 - 16985798.96312772) / 15656809.254191486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OrderByCustomComparer - Duration of single invocation 1.36 ms 1.49 ms 1.10 0.03 False
OrderByValueType - Duration of single invocation 883.56 μs 950.29 μs 1.08 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_OrderBy.OrderByCustomComparer(NumberOfPeople: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4907450836038962 > 1.4206957934244793.
IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.681170431924336 (T) = (0 -1486098.102197795) / Math.Sqrt((53397989.32957973 / (19)) + (713321107.3633078 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09688743041712485 = (1354831.9189259566 - 1486098.102197795) / 1354831.9189259566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 950.2946151018103 > 936.8804832131411.
IsChangePoint: Marked as a change because one of 11/7/2022 8:57:30 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.319557448073876 (T) = (0 -951531.4647466947) / Math.Sqrt((111542567.16488154 / (19)) + (275175982.43252283 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05407162566171261 = (902719.9305828515 - 951531.4647466947) / 902719.9305828515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 1.99 μs 2.13 μs 1.07 0.04 False
IsMatch - Duration of single invocation 1.97 μs 2.16 μs 1.10 0.03 False
IsMatch - Duration of single invocation 2.01 μs 2.20 μs 1.09 0.04 False
IsMatch - Duration of single invocation 1.76 μs 1.87 μs 1.06 0.04 False
IsMatch - Duration of single invocation 1.97 μs 2.18 μs 1.11 0.04 False
IsMatch - Duration of single invocation 1.51 μs 1.63 μs 1.08 0.07 False
IsMatch - Duration of single invocation 7.14 μs 7.60 μs 1.06 0.04 False
IsMatch - Duration of single invocation 1.78 μs 1.90 μs 1.07 0.05 False
IsMatch - Duration of single invocation 2.02 μs 2.19 μs 1.09 0.05 False
IsMatch - Duration of single invocation 1.97 μs 2.26 μs 1.15 0.04 False
IsMatch - Duration of single invocation 2.02 μs 2.19 μs 1.08 0.04 False
IsMatch - Duration of single invocation 7.14 μs 7.74 μs 1.08 0.05 False
IsMatch - Duration of single invocation 2.01 μs 2.18 μs 1.08 0.04 False
IsMatch - Duration of single invocation 2.05 μs 2.21 μs 1.08 0.02 False
IsMatch - Duration of single invocation 1.77 μs 1.87 μs 1.06 0.05 False
IsMatch - Duration of single invocation 1.52 μs 1.64 μs 1.08 0.03 False
IsMatch - Duration of single invocation 2.05 μs 2.27 μs 1.11 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.132926700178424 > 2.0944272641612343.
IsChangePoint: Marked as a change because one of 10/12/2022 7:36:52 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.128727457079805 (T) = (0 -2174.1030274136792) / Math.Sqrt((621.6540037947839 / (19)) + (1986.2522714872093 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08720777295917831 = (1999.712549420221 - 2174.1030274136792) / 1999.712549420221 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1598921259398494 > 2.1097846724376565.
IsChangePoint: Marked as a change because one of 10/7/2022 12:26:39 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.51121942302988 (T) = (0 -2177.49900600978) / Math.Sqrt((997.3346836004439 / (19)) + (1072.8419303920748 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07788493205660336 = (2020.1590552482387 - 2177.49900600978) / 2020.1590552482387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.198871615786772 > 2.1451648013830917.
IsChangePoint: Marked as a change because one of 10/4/2022 5:30:44 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.65511864841503 (T) = (0 -2189.3773508252175) / Math.Sqrt((889.8729734592279 / (19)) + (1698.4217319252627 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08562438723646967 = (2016.6987556335444 - 2189.3773508252175) / 2016.6987556335444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.87373194246456 > 1.8291563893807201.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.975537565479017 (T) = (0 -1871.6120580262339) / Math.Sqrt((483.5801191755797 / (19)) + (1293.553751334011 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07264944385941097 = (1744.849697858549 - 1871.6120580262339) / 1744.849697858549 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1846140694910514 > 2.091739311187572.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.723013708769333 (T) = (0 -2171.9443107068137) / Math.Sqrt((970.365694031696 / (19)) + (1650.925059818667 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08786222489852002 = (1996.5251674304814 - 2171.9443107068137) / 1996.5251674304814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6286421741815398 > 1.6025137632749347.
IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.341506517876477 (T) = (0 -1647.734898860627) / Math.Sqrt((563.4205464248238 / (19)) + (871.7374550509701 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07280015581762163 = (1535.919705012371 - 1647.734898860627) / 1535.919705012371 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.601129172728741 > 7.521126704817818.
IsChangePoint: Marked as a change because one of 9/30/2022 2:28:26 AM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.295923434528227 (T) = (0 -7681.920634764331) / Math.Sqrt((17281.12990566542 / (18)) + (18890.892476217272 / (28))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (28) - 2, .025) and -0.05729485438921182 = (7265.637019677067 - 7681.920634764331) / 7265.637019677067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9011551991055304 > 1.8281485706537264.
IsChangePoint: Marked as a change because one of 10/7/2022 11:47:58 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.718536762544382 (T) = (0 -1886.8887492257345) / Math.Sqrt((1149.73821673168 / (19)) + (2825.3345195953207 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07088296535616333 = (1761.993429971292 - 1886.8887492257345) / 1761.993429971292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.192890500697837 > 2.1282467293591614.
IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.426559669522248 (T) = (0 -2187.5608998860903) / Math.Sqrt((1199.6956226781385 / (19)) + (2285.3835035367597 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08618566775382709 = (2013.984316705124 - 2187.5608998860903) / 2013.984316705124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2624976342513317 > 2.092094407551897.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.670239738759296 (T) = (0 -2182.2894033691077) / Math.Sqrt((1457.0907389350173 / (19)) + (1985.1567596286438 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08893027658739515 = (2004.0671568139303 - 2182.2894033691077) / 2004.0671568139303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1946836150070124 > 2.115538140842655.
IsChangePoint: Marked as a change because one of 10/4/2022 10:23:05 PM, 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.35358740172079 (T) = (0 -2181.9646762149255) / Math.Sqrt((2482.6903729548753 / (22)) + (275.84961634919074 / (25))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (25) - 2, .025) and -0.07900273059517339 = (2022.204962364982 - 2181.9646762149255) / 2022.204962364982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.7378158786593465 > 7.5305497716430665.
IsChangePoint: Marked as a change because one of 10/4/2022 3:13:24 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.773468733043508 (T) = (0 -7654.430585631961) / Math.Sqrt((20283.11730928976 / (19)) + (9076.101873401976 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.05541609432298522 = (7252.524029910712 - 7654.430585631961) / 7252.524029910712 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.178042576394596 > 2.1479502223135096.
IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.077382931296203 (T) = (0 -2189.794787139382) / Math.Sqrt((1377.5883426081487 / (19)) + (1357.4512675028757 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07609101358075164 = (2034.953140118437 - 2189.794787139382) / 2034.953140118437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2081701582472526 > 2.129683870959425.
IsChangePoint: Marked as a change because one of 10/5/2022 8:25:09 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.58684117169094 (T) = (0 -2200.6903232961613) / Math.Sqrt((891.8014750670341 / (19)) + (1564.4851751468368 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0827470206292925 = (2032.5064686090018 - 2200.6903232961613) / 2032.5064686090018 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.873429242695289 > 1.8248559903671862.
IsChangePoint: Marked as a change because one of 10/2/2022 5:38:34 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.256977733573375 (T) = (0 -1872.7436031363807) / Math.Sqrt((682.8962602067835 / (19)) + (1812.9314766514688 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07029086745428731 = (1749.7520160951635 - 1872.7436031363807) / 1749.7520160951635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6380556526190901 > 1.6164680554623867.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.657818763048219 (T) = (0 -1645.4029590063046) / Math.Sqrt((606.8863153320568 / (19)) + (1506.1953082661826 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.07107731113694034 = (1536.2130650118263 - 1645.4029590063046) / 1536.2130650118263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2727688570856057 > 2.216945432117046.
IsChangePoint: Marked as a change because one of 10/1/2022 12:04:49 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.396253637156024 (T) = (0 -2269.979542963806) / Math.Sqrt((1659.2549698551577 / (19)) + (2404.364232167333 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07074562944640822 = (2119.9988872589843 - 2269.979542963806) / 2119.9988872589843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 1.64 μs 1.76 μs 1.07 0.07 False
IsNaN - Duration of single invocation 7.02 ms 8.01 ms 1.14 0.04 False
ToString - Duration of single invocation 1.63 μs 1.76 μs 1.08 0.03 False
ToStringWithFormat - Duration of single invocation 1.62 μs 1.74 μs 1.08 0.06 False
ToStringWithFormat - Duration of single invocation 1.65 μs 1.82 μs 1.11 0.08 False
IsNaN - Duration of single invocation 7.01 ms 7.99 ms 1.14 0.08 False
ToStringWithFormat - Duration of single invocation 1.68 μs 1.78 μs 1.06 0.02 False
ToStringWithCultureInfo - Duration of single invocation 1.65 μs 1.78 μs 1.08 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7578910365196643 > 1.7212146660668837.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.75685604840131 (T) = (0 -1755.4914071482588) / Math.Sqrt((1331.265963884527 / (19)) + (1125.7199981166286 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.0687704009321122 = (1642.5337056651579 - 1755.4914071482588) / 1642.5337056651579 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.IsNaN(value: NaN)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.010510463541667 > 7.367699549947917.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -24.81700497921273 (T) = (0 -7930858.119793643) / Math.Sqrt((45383136.92971627 / (19)) + (37891083226.29864 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13021556792158082 = (7017119.870661629 - 7930858.119793643) / 7017119.870661629 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.76195346636227 > 1.7272481285491412.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -14.320322014152286 (T) = (0 -1767.0824277711413) / Math.Sqrt((373.1027976158298 / (19)) + (1137.8609812207633 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06714069628468843 = (1655.903887766009 - 1767.0824277711413) / 1655.903887766009 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7449876436105751 > 1.72777717439529.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.821526101481126 (T) = (0 -1763.8323810006407) / Math.Sqrt((591.1290131969181 / (19)) + (862.7535953959755 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06067305121187224 = (1662.9369238573315 - 1763.8323810006407) / 1662.9369238573315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.821695667360285 > 1.73047359339664.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.294395391633852 (T) = (0 -1773.9161323341434) / Math.Sqrt((307.92047568756647 / (19)) + (926.0301758258867 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06892600027743205 = (1659.53127894142 - 1773.9161323341434) / 1659.53127894142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.IsNaN(value: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.989730792410715 > 7.3688997078125.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.79664818100782 (T) = (0 -7935504.834861425) / Math.Sqrt((11179563515.566446 / (19)) + (40696308301.264595 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12704875484657985 = (7040959.675202028 - 7935504.834861425) / 7040959.675202028 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7809831645575716 > 1.7685952325148044.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.944102094642307 (T) = (0 -1806.52928237067) / Math.Sqrt((1070.484134024198 / (18)) + (419.9661606318722 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.060284934312406876 = (1703.814912302042 - 1806.52928237067) / 1703.814912302042 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7770752064220183 > 1.7316965801968094.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -13.631998511534707 (T) = (0 -1758.6322483370125) / Math.Sqrt((672.577495583565 / (19)) + (563.7299148737067 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.061305085870773345 = (1657.0468489690693 - 1758.6322483370125) / 1657.0468489690693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ILogB - Duration of single invocation 115.73 μs 123.20 μs 1.06 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Double.ILogB

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.19840400132276 > 121.0068327795129.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.91530693384087 (T) = (0 -123830.96768890494) / Math.Sqrt((2521631.197985992 / (19)) + (3732860.3311853404 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07579651098006931 = (115106.31092872088 - 123830.96768890494) / 115106.31092872088 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 20.34 μs 21.95 μs 1.08 0.22 False
List - Duration of single invocation 12.27 μs 13.85 μs 1.13 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.94967335766423 > 21.576032141366426.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.298709645723354 (T) = (0 -22191.104635847274) / Math.Sqrt((169956.41122827298 / (19)) + (263701.86329011945 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08120049086351062 = (20524.50477350796 - 22191.104635847274) / 20524.50477350796 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSet&lt;String&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.850488501535454 > 12.859630571409074.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -8.283309095170305 (T) = (0 -13147.622147120012) / Math.Sqrt((118070.9008522012 / (19)) + (80466.66081568776 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06389880565400857 = (12357.963066833032 - 13147.622147120012) / 12357.963066833032 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryGetValue_17_Int_32ByteRefsValue - Duration of single invocation 220.05 ns 247.75 ns 1.13 0.05 False
ContainsKey_3k_Int_32ByteValue - Duration of single invocation 180.62 ns 200.33 ns 1.11 0.05 False
ContainsKey_17_Int_32ByteRefsValue - Duration of single invocation 177.77 ns 205.54 ns 1.16 0.05 False
ContainsKey_3k_Int_32ByteRefsValue - Duration of single invocation 178.43 ns 200.35 ns 1.12 0.08 False
ContainsKey_17_Int_32ByteValue - Duration of single invocation 180.84 ns 204.08 ns 1.13 0.09 False
TryGetValue_3k_Int_32ByteValue - Duration of single invocation 193.98 ns 221.93 ns 1.14 0.08 False
ContainsValue_3k_Int_Int - Duration of single invocation 176.85 ns 204.03 ns 1.15 0.07 False
ContainsValue_17_Int_Int - Duration of single invocation 179.57 ns 206.56 ns 1.15 0.07 False
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation 216.21 ns 246.86 ns 1.14 0.06 False
TryGetValue_3k_Int_Int - Duration of single invocation 179.43 ns 207.63 ns 1.16 0.08 False
TryGetValue_17_Int_32ByteValue - Duration of single invocation 194.74 ns 217.57 ns 1.12 0.08 False
TryGetValue_17_Int_Int - Duration of single invocation 186.55 ns 210.72 ns 1.13 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteRefsValue

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 247.7470547240009 > 235.93864793510443.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.256360603254665 (T) = (0 -248.0469825861785) / Math.Sqrt((39.07598690154577 / (19)) + (26.632464255237135 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11940669445435767 = (221.58790349836727 - 248.0469825861785) / 221.58790349836727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_3k_Int_32ByteValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.33167038690476 > 186.30829313038794.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -34.30720676021228 (T) = (0 -202.3991781545294) / Math.Sqrt((4.835884639270098 / (19)) + (9.165033788774085 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1484937258057682 = (176.2301121954574 - 202.3991781545294) / 176.2301121954574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_17_Int_32ByteRefsValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 205.54483200617443 > 187.71843638077073.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -25.45682606861233 (T) = (0 -205.70219954571925) / Math.Sqrt((10.587878525022155 / (19)) + (18.387516023428976 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.15788119025884956 = (177.653978038743 - 205.70219954571925) / 177.653978038743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_3k_Int_32ByteRefsValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.34848830128206 > 187.96361224870137.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -19.61190021991931 (T) = (0 -203.52695006447283) / Math.Sqrt((14.612295169038374 / (19)) + (24.63288587802347 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14120395596640842 = (178.34406286483613 - 203.52695006447283) / 178.34406286483613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_17_Int_32ByteValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 204.07876555725247 > 186.76869152895884.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -25.197672904686367 (T) = (0 -205.76023577147092) / Math.Sqrt((7.070190274662941 / (19)) + (24.59050909033985 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1586612516777041 = (177.58446264905876 - 205.76023577147092) / 177.58446264905876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 221.92876675724636 > 203.29720387731484.
IsChangePoint: Marked as a change because one of 9/28/2022 10:18:57 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.919816498822197 (T) = (0 -221.58763846577193) / Math.Sqrt((14.474046414604324 / (19)) + (19.073678495441147 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.13485397036015542 = (195.25652132621894 - 221.58763846577193) / 195.25652132621894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.ContainsValue_3k_Int_Int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 204.02523586309522 > 185.50251220366383.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.08444322691271 (T) = (0 -202.5049348813606) / Math.Sqrt((16.114267094196272 / (19)) + (12.055513506617784 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14797800633280095 = (176.40140644180082 - 202.5049348813606) / 176.40140644180082 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.ContainsValue_17_Int_Int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 206.5598594039606 > 186.38903918784467.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -27.41758747655611 (T) = (0 -205.89166736905716) / Math.Sqrt((6.897376838586552 / (19)) + (19.362074437160047 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.15840919495903985 = (177.7365617132703 - 205.89166736905716) / 177.7365617132703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 246.86020042989418 > 229.81479915364585.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -27.809286013722417 (T) = (0 -244.97777321511867) / Math.Sqrt((8.315433305268265 / (19)) + (12.40172650960683 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11922375756307577 = (218.88185589315685 - 244.97777321511867) / 218.88185589315685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 207.63204005952383 > 189.61195145070042.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 1:45:10 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.632891893205866 (T) = (0 -206.77772337003606) / Math.Sqrt((27.199164448395063 / (19)) + (6.071296636319607 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14693378029829496 = (180.2874123353985 - 206.77772337003606) / 180.2874123353985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteValue

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 217.57473413658303 > 204.77623238284852.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.37614034896514 (T) = (0 -225.29231625402645) / Math.Sqrt((13.363649102069335 / (19)) + (28.899596862165904 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1428505668656786 = (197.1319110178168 - 225.29231625402645) / 197.1319110178168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.7196657828833 > 189.37357058115322.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 10:22:58 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.803485877410424 (T) = (0 -209.33600083298393) / Math.Sqrt((30.262146201116405 / (19)) + (12.798858092486542 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.14758413060580364 = (182.41451345486675 - 209.33600083298393) / 182.41451345486675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in GuardedDevirtualization.ThreeClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 17.43 ns 19.25 ns 1.10 0.12 False
Call - Duration of single invocation 17.08 ns 19.12 ns 1.12 0.09 False
Call - Duration of single invocation 17.20 ns 18.98 ns 1.10 0.06 False
Call - Duration of single invocation 17.08 ns 18.96 ns 1.11 0.08 False
Call - Duration of single invocation 17.04 ns 18.94 ns 1.11 0.07 False
Call - Duration of single invocation 17.09 ns 19.34 ns 1.13 0.07 False
Call - Duration of single invocation 17.09 ns 18.88 ns 1.11 0.07 False
Call - Duration of single invocation 17.17 ns 19.17 ns 1.12 0.05 False
Call - Duration of single invocation 17.20 ns 19.93 ns 1.16 0.13 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassVirtual*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.00)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.252728202351484 > 18.084687455244403.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.219009962545947 (T) = (0 -19.046347382231794) / Math.Sqrt((0.1427262635095588 / (19)) + (0.096363450450927 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10450573468562442 = (17.24422679222481 - 19.046347382231794) / 17.24422679222481 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.67)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.122081262449022 > 17.987550887517813.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -42.32672981152475 (T) = (0 -18.980022748085023) / Math.Sqrt((0.018047553424973294 / (19)) + (0.027760971130146188 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10896471859721381 = (17.11508258990766 - 18.980022748085023) / 17.11508258990766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.98132671454451 > 18.03258270398557.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -31.023765073990994 (T) = (0 -19.01324808749855) / Math.Sqrt((0.028949120256357746 / (19)) + (0.0577430320726923 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10829046241220315 = (17.155473887338218 - 19.01324808749855) / 17.155473887338218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=1.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.963037177532833 > 17.93577533034754.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.954054335885072 (T) = (0 -18.990166962779806) / Math.Sqrt((0.06201587938188707 / (19)) + (0.06784976764361328 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10512630512768835 = (17.183707305370536 - 18.990166962779806) / 17.183707305370536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.93508362766175 > 18.025835516261882.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.345054287217902 (T) = (0 -19.051592943924188) / Math.Sqrt((0.0968971542778751 / (19)) + (0.1646391482634655 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10545925957178841 = (17.234097755266014 - 19.051592943924188) / 17.234097755266014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.341917386428708 > 17.95185572973713.
IsChangePoint: Marked as a change because one of 7/27/2022 2:59:41 AM, 7/27/2022 12:09:41 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -42.52857645489969 (T) = (0 -18.994742989987518) / Math.Sqrt((0.014540435306101595 / (19)) + (0.0333972840888933 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10996885023644302 = (17.112861307721655 - 18.994742989987518) / 17.112861307721655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.67)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.88379756097561 > 17.94849205828998.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -30.350586306645738 (T) = (0 -18.920996207507926) / Math.Sqrt((0.0523185786152531 / (19)) + (0.014738510201632407 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10116015547136449 = (17.182783188707525 - 18.920996207507926) / 17.182783188707525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.16515075677111 > 18.011545995059357.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -21.898730399615467 (T) = (0 -19.068910587892145) / Math.Sqrt((0.04164847079185644 / (19)) + (0.14342698112425858 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10891318506223356 = (17.196035582192103 - 19.068910587892145) / 17.196035582192103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=1.00 pD=0.00)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.925088852242748 > 17.924351179559423.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -25.712305864259076 (T) = (0 -19.123027996004847) / Math.Sqrt((0.01880128562756793 / (19)) + (0.14582529938044594 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11838206189597329 = (17.09883290115179 - 19.123027996004847) / 17.09883290115179 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NoArguments - Duration of single invocation 177.80 ns 201.37 ns 1.13 0.25 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.NoArguments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.37286708581664 > 192.44965567429796.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -11.401005638928016 (T) = (0 -208.11906521311514) / Math.Sqrt((38.052167295675595 / (19)) + (51.94690045887416 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.12057363704415015 = (185.72546982462637 - 208.11906521311514) / 185.72546982462637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added refs/heads/main untriaged New issue has not been triaged by the area owner labels Nov 22, 2022
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Cache

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 588.05 ms 626.36 ms 1.07 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 400000, unique: 1, cacheSize: 15)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 626.3597338666667 > 623.2510099550002.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.254099164047604 (T) = (0 -636950437.2334772) / Math.Sqrt((75703319952527.66 / (19)) + (103710594856784.27 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07112762311049983 = (594654104.2269135 - 636950437.2334772) / 594654104.2269135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in FractalPerf.Launch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 3.02 secs 3.40 secs 1.12 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'FractalPerf.Launch*'

Payloads

Baseline
Compare

Histogram

FractalPerf.Launch.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.399903676285714 > 3.1731409847062504.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.947999073863111 (T) = (0 -3357161628.6985617) / Math.Sqrt((8509908708958263 / (19)) + (19851685541862440 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08757687290696689 = (3086826975.0213223 - 3357161628.6985617) / 3086826975.0213223 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Numerics.Tests.Perf_Plane

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotCoordinateBenchmark - Duration of single invocation 32.72 ns 34.83 ns 1.06 0.30 False
DotNormalBenchmark - Duration of single invocation 31.24 ns 34.23 ns 1.10 0.25 False
EqualsBenchmark - Duration of single invocation 69.29 ns 79.70 ns 1.15 0.20 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Plane.DotCoordinateBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.82734148222444 > 34.53252192896299.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.570731824623445 (T) = (0 -36.93862038380261) / Math.Sqrt((1.1555028717301932 / (19)) + (2.109647419980061 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11806372951406516 = (33.038027626436765 - 36.93862038380261) / 33.038027626436765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Plane.DotNormalBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.234984615551404 > 32.937761997258626.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/16/2022 11:33:39 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.630872561044286 (T) = (0 -34.931392443092356) / Math.Sqrt((3.2632256244223696 / (24)) + (2.1808093195518548 / (23))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (23) - 2, .025) and -0.10034301924604579 = (31.74591180396393 - 34.931392443092356) / 31.74591180396393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Plane.EqualsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.69509885371339 > 72.20722000752143.
IsChangePoint: Marked as a change because one of 10/6/2022 8:29:49 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.046076395871275 (T) = (0 -82.42063595738595) / Math.Sqrt((10.075800130488949 / (19)) + (4.464955552188108 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.2073898521313055 = (68.26348243021557 - 82.42063595738595) / 68.26348243021557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForEach - Duration of single invocation 1.98 μs 2.08 μs 1.05 0.01 False
ForEach - Duration of single invocation 1.98 μs 2.13 μs 1.07 0.01 False
ForEach - Duration of single invocation 794.90 ns 851.84 ns 1.07 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.081011802232855 > 2.0464303040153835.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -20.661911712058103 (T) = (0 -2110.3014339552637) / Math.Sqrt((300.7125937626384 / (19)) + (988.2956414263572 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07527575031697202 = (1962.5676793447492 - 2110.3014339552637) / 1962.5676793447492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.125406386931382 > 2.074613750702666.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.769970260066547 (T) = (0 -2107.3642803694884) / Math.Sqrt((487.6467854405016 / (19)) + (738.7643424432172 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06867188619235733 = (1971.9469629522657 - 2107.3642803694884) / 1971.9469629522657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 851.8424417816683 > 842.306815332955.
IsChangePoint: Marked as a change because one of 9/30/2022 2:28:26 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.018781221067503 (T) = (0 -852.4527084056534) / Math.Sqrt((165.84413320349853 / (19)) + (311.9231607691492 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.05588695309240112 = (807.3333096019938 - 852.4527084056534) / 807.3333096019938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Tests.Add_Remove_SteadyState<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableStack - Duration of single invocation 90.31 ns 100.91 ns 1.12 0.11 False
ImmutableQueue - Duration of single invocation 329.58 ns 358.38 ns 1.09 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Add_Remove_SteadyState<String>.ImmutableStack(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 100.91298776046965 > 97.0512710987129.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -7.682964809178396 (T) = (0 -100.2294166282283) / Math.Sqrt((12.673249127301307 / (19)) + (2.9662365544371574 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07226205655465577 = (93.47473970148766 - 100.2294166282283) / 93.47473970148766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Add_Remove_SteadyState&lt;String&gt;.ImmutableQueue(Count: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 358.3839946218228 > 346.05484232228326.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -25.472026864037556 (T) = (0 -362.2621538647403) / Math.Sqrt((17.09823585172564 / (19)) + (22.0107682310481 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.10047303074285055 = (329.18767088749377 - 362.2621538647403) / 329.18767088749377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 3.16 ms 3.43 ms 1.09 0.06 False
ReadLineAsync - Duration of single invocation 3.47 ms 3.73 ms 1.07 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.432416090178571 > 3.37246304390625.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -9.674646020897889 (T) = (0 -3492239.939717916) / Math.Sqrt((7006654579.611719 / (19)) + (4862104657.5793 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06897056056447716 = (3266918.7240047236 - 3492239.939717916) / 3266918.7240047236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.725787465178571 > 3.598594877671875.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.098560357100265 (T) = (0 -3800653.8742578174) / Math.Sqrt((5474650463.709884 / (19)) + (7430646231.4796095 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11068426585855518 = (3421903.0475955512 - 3800653.8742578174) / 3421903.0475955512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 853.02 μs 997.93 μs 1.17 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Fib.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 997.9263931361606 > 870.3464999620446.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -18.195495818322485 (T) = (0 -996832.8699357354) / Math.Sqrt((788389937.347832 / (19)) + (999146626.6376448 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.19098258036748592 = (836983.5851236008 - 996832.8699357354) / 836983.5851236008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 1.59 μs 2.02 μs 1.27 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.01974460139864 > 1.6808345949514591.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -10.212223453228717 (T) = (0 -2043.8522113359127) / Math.Sqrt((16822.211902853356 / (19)) + (19279.47869949759 / (27))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (27) - 2, .025) and -0.2497296449926534 = (1635.4354876073437 - 2043.8522113359127) / 1635.4354876073437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Buffers.Tests.NonStandardArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RentNoReturn - Duration of single invocation 343.34 ns 380.24 ns 1.11 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.NonStandardArrayPoolTests<Byte>.RentNoReturn(RentalSize: 64, UseSharedPool: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 380.2445931632654 > 359.17924860000005.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 9:30:44 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -15.497798606056637 (T) = (0 -370.29542485751506) / Math.Sqrt((31.20141979650934 / (19)) + (29.927560775698844 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07401110990405466 = (344.77802086292655 - 370.29542485751506) / 344.77802086292655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 106.61 μs 113.51 μs 1.06 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.50810538069544 > 111.0253787145688.
IsChangePoint: Marked as a change because one of 9/29/2022 2:35:34 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -4.158437100953634 (T) = (0 -114120.57446127885) / Math.Sqrt((37904132.06282913 / (19)) + (4802371.935338787 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.056673923179429715 = (107999.80198044547 - 114120.57446127885) / 107999.80198044547 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 8.39 ms 9.06 ms 1.08 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.063683086419754 > 8.847667130271883.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -17.343853525769905 (T) = (0 -9120593.61291954) / Math.Sqrt((10229339904.954676 / (19)) + (25935955847.941914 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.07848887825732781 = (8456826.766407657 - 9120593.61291954) / 8456826.766407657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.Concurrent.Count<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 1.60 μs 1.97 μs 1.23 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<String>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9719180123088518 > 1.6780411145001575.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -27.189897852008553 (T) = (0 -2001.8631925076954) / Math.Sqrt((124.31084644083165 / (19)) + (6113.515583387627 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.2557785272272327 = (1594.1212157272848 - 2001.8631925076954) / 1594.1212157272848 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline dd871a350bff581b6d313e88b8f29043a9772d18
Compare 715fde2b08357dfb72248f18ab09dd27cf7f2888
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 12.56 μs 13.63 μs 1.09 0.12 False
Queue - Duration of single invocation 21.81 μs 28.57 μs 1.31 0.05 False
ICollection - Duration of single invocation 14.34 μs 15.68 μs 1.09 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.63272268962511 > 13.200929927276938.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.342130183988482 (T) = (0 -13694.105557643186) / Math.Sqrt((40470.123098049335 / (19)) + (156675.04073090976 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08603277018101171 = (12609.293138880843 - 13694.105557643186) / 12609.293138880843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.AddGivenSize&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.57377286546944 > 22.972141471119972.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -12.147597143316778 (T) = (0 -24599.332864948832) / Math.Sqrt((166072.9011735526 / (19)) + (1061659.1811166515 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.11940209191361009 = (21975.421560001236 - 24599.332864948832) / 21975.421560001236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.AddGivenSize&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.675097669846052 > 15.058248851316096.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -23.500097371700242 (T) = (0 -15647.540794917408) / Math.Sqrt((31746.470149640278 / (19)) + (27126.65791642167 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08361285120733085 = (14440.158011676735 - 15647.540794917408) / 14440.158011676735 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Nov 22, 2022

@kg the commit range is 3cfd6cf...ce58915 which points to the jiterpreter regressing non-wasm cases here, any idea why?

@kotlarmilos
Copy link
Member

@kg regressions in ce58915

@kotlarmilos kotlarmilos removed the untriaged New issue has not been triaged by the area owner label Nov 23, 2022
@kotlarmilos kotlarmilos mentioned this issue Nov 23, 2022
Closed
@radekdoulik
Copy link
Member

the individual links here don't work, so I wonder if that might be infra issue

@radekdoulik
Copy link
Member

radekdoulik commented Nov 23, 2022 •
edited
Loading

I can see the regression in our measurements too. Interestingly it affects only chrome and not firefox. perf tool
obrazek

I will schedule measurement for the commit before ce58915 to confirm it.

@kg
Copy link
Member

kg commented Nov 23, 2022

I can see the regression in our measurements too. Interestingly it affects only chrome and not firefox. perf tool obrazek

I will schedule measurement for the commit before dotnet/runtime@ce58915 to confirm it.

Odd, this regression report says it is for Linux and not WASM, though?

@radekdoulik
Copy link
Member

radekdoulik commented Nov 24, 2022 •
edited
Loading

Odd, this regression report says it is for Linux and not WASM, though?

yes, this issue report is from desktop measurements. I can see it in our browser-bench measurements too, that's what is in the screenshots from the performance tool.

@radekdoulik
Copy link
Member

The scheduled f652776 measurement confirmed the regression - Span, Reverse chars. That can be used to reproduce the issue on the chrome browser. There are also nice improvements in other areas - Span, Index of bytes. It is visible in the perf tool

Regression example
obrazek

Improvement example
obrazek

@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Nov 24, 2022
@kotlarmilos kotlarmilos removed the untriaged New issue has not been triaged by the area owner label Nov 24, 2022
@kotlarmilos kotlarmilos transferred this issue from dotnet/perf-autofiling-issues Nov 24, 2022
@kotlarmilos
Copy link
Member

Fixed by 5a7d46b

@jeffhandley jeffhandley added os-linux Linux OS (any supported distro) arch-x64 and removed refs/heads/main labels Dec 28, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Jan 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@kg kg

Labels
arch-x64 area-Codegen-meta-mono os-linux Linux OS (any supported distro)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@lewing @kg @jeffhandley @radekdoulik @jeffschwMSFT @kotlarmilos