[Perf] Changes at 5/30/2022 5:18:40 PM #5818

performanceautofiler · 2022-06-07T12:20:45Z

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	192.96 ns	213.60 ns	1.11	0.01	True
Parse - Duration of single invocation	195.49 ns	219.46 ns	1.12	0.01	True
Parse - Duration of single invocation	198.31 ns	221.08 ns	1.11	0.02	False
TryParse - Duration of single invocation	191.25 ns	209.77 ns	1.10	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 213.59818709477662 > 202.9408767597273.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -33.8455448117254 (T) = (0 -216.4364716087262) / Math.Sqrt((3.3933441817344323 / (20)) + (11.474042588122181 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.1096263170980368 = (195.05347725959152 - 216.4364716087262) / 195.05347725959152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.45801682228233 > 205.51840818523456.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -70.37129440561424 (T) = (0 -218.19961195324342) / Math.Sqrt((1.2040297186807147 / (21)) + (1.8972982421737523 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.11105479217887208 = (196.38960516549827 - 218.19961195324342) / 196.38960516549827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 221.08407705675836 > 207.70304896410804.
IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 4/8/2022 8:59:46 PM, 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -107.58771328361154 (T) = (0 -219.89926608373258) / Math.Sqrt((0.26625305240315783 / (20)) + (1.3930438033601196 / (49))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (49) - 2, .025) and -0.11106169941099465 = (197.91814100000695 - 219.89926608373258) / 197.91814100000695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 209.77342719621464 > 200.79379386477962.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -35.332706796885425 (T) = (0 -213.0251850818254) / Math.Sqrt((5.065105868154211 / (21)) + (4.634449194429837 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.10599820485001284 = (192.60897906313897 - 213.0251850818254) / 192.60897906313897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in PerfLabTests.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IntObj - Duration of single invocation	311.85 μs	408.23 μs	1.31	0.10	False
ObjrefValueTypeObj - Duration of single invocation	311.84 μs	408.16 μs	1.31	0.09	False
FooObjCastIfIsa - Duration of single invocation	498.94 μs	540.89 μs	1.08	0.09	False
ScalarValueTypeObj - Duration of single invocation	311.83 μs	409.89 μs	1.31	0.10	False
FooObjIsFoo2 - Duration of single invocation	311.99 μs	407.57 μs	1.31	0.09	False
FooObjIsFoo - Duration of single invocation	311.83 μs	407.29 μs	1.31	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.IntObj

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 408.22912087912084 > 327.4477657781863.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/18/2022 2:31:51 PM, 5/9/2022 5:23:46 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -89.1498366369618 (T) = (0 -408451.78620079975) / Math.Sqrt((23269192.631297983 / (21)) + (1940946.9459814706 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.30518203719850434 = (312946.22095590376 - 408451.78620079975) / 312946.22095590376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.ObjrefValueTypeObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 408.1571367521368 > 327.4499041053921.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/18/2022 2:31:51 PM, 5/9/2022 5:23:46 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -169.73512437067848 (T) = (0 -408771.4794091016) / Math.Sqrt((5537223.312989423 / (21)) + (2999946.7592564807 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.3085691730883711 = (312380.4899395228 - 408771.4794091016) / 312380.4899395228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.FooObjCastIfIsa

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 540.8852155172414 > 523.8883227539062.
IsChangePoint: Marked as a change because one of 4/21/2022 5:54:30 PM, 4/27/2022 1:24:17 PM, 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -9.434143163154937 (T) = (0 -541121.1360659633) / Math.Sqrt((293937886.24740076 / (20)) + (13818207.842003545 / (49))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (49) - 2, .025) and -0.07235820478160508 = (504608.5661051731 - 541121.1360659633) / 504608.5661051731 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.ScalarValueTypeObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 409.8892653508772 > 327.43123943014706.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/18/2022 2:31:51 PM, 5/9/2022 5:23:46 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -49.44157056408767 (T) = (0 -410271.15956712543) / Math.Sqrt((78997898.1695117 / (21)) + (2329287.4366033617 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.3074894269260242 = (313785.4510469689 - 410271.15956712543) / 313785.4510469689 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.FooObjIsFoo2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 407.5682234432235 > 327.43686661010554.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/18/2022 2:31:51 PM, 5/9/2022 5:23:46 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -193.60855858424586 (T) = (0 -408553.002172653) / Math.Sqrt((4563118.1826357 / (21)) + (1433305.1663362952 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.3072249299909734 = (312534.58589982224 - 408553.002172653) / 312534.58589982224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.FooObjIsFoo

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 407.29319526627216 > 327.4299144513575.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/18/2022 6:18:02 PM, 5/9/2022 5:23:46 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -327.4379234354499 (T) = (0 -408519.7918590081) / Math.Sqrt((192.5777493720675 / (21)) + (4445309.474067671 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.3100133107804741 = (311844.0007419634 - 408519.7918590081) / 311844.0007419634 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	291.02 ns	346.01 ns	1.19	0.08	False
SingleWithPredicate_FirstElementMatches - Duration of single invocation	709.86 ns	755.78 ns	1.06	0.03	False
Sum - Duration of single invocation	460.28 ns	494.31 ns	1.07	0.01	False
Take_All - Duration of single invocation	998.74 ns	1.10 μs	1.10	0.06	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 346.0126568150364 > 304.54503046974804.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -15.911161415692309 (T) = (0 -345.7416393922307) / Math.Sqrt((153.10417369544206 / (21)) + (16.033060155854994 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.1453593448744747 = (301.8630274764311 - 345.7416393922307) / 301.8630274764311 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 755.784053055276 > 745.4713661679579.
IsChangePoint: Marked as a change because one of 4/2/2022 7:01:35 PM, 5/30/2022 12:57:05 PM, 5/31/2022 9:21:33 PM, 6/6/2022 10:51:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -14.291199728469122 (T) = (0 -756.369553540537) / Math.Sqrt((1245.9591334122474 / (67)) + (5.459722731067346 / (5))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (5) - 2, .025) and -0.09150964788679382 = (692.9572771114746 - 756.369553540537) / 692.9572771114746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 494.310539067553 > 483.2967311391723.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -55.25566561298477 (T) = (0 -493.02233498731624) / Math.Sqrt((3.4671954685018265 / (21)) + (9.758231062621372 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.07254845813273189 = (459.67371567122507 - 493.02233498731624) / 459.67371567122507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.101625379885504 > 1048.0686444655432.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/24/2022 7:55:26 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -30.793958827921834 (T) = (0 -1140.9300339015256) / Math.Sqrt((55.56510204176256 / (19)) + (871.1461026956437 / (51))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (51) - 2, .025) and -0.1372959393209459 = (1003.1953816547956 - 1140.9300339015256) / 1003.1953816547956 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2022-06-07T12:20:49Z

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	85.97 μs	99.15 μs	1.15	0.06	False
Count - Duration of single invocation	120.81 μs	137.52 μs	1.14	0.03	False
Count - Duration of single invocation	60.07 μs	74.28 μs	1.24	0.18	False
Count - Duration of single invocation	162.98 μs	182.11 μs	1.12	0.02	False
Count - Duration of single invocation	65.50 μs	80.79 μs	1.23	0.12	False
Count - Duration of single invocation	78.01 μs	91.12 μs	1.17	0.24	False
Count - Duration of single invocation	62.43 μs	74.73 μs	1.20	0.17	False
Count - Duration of single invocation	60.67 μs	74.64 μs	1.23	0.11	False
Count - Duration of single invocation	93.43 μs	102.83 μs	1.10	0.14	False
Count - Duration of single invocation	134.27 μs	154.73 μs	1.15	0.02	False
Count - Duration of single invocation	53.24 μs	64.37 μs	1.21	0.20	False
Count - Duration of single invocation	72.88 μs	92.84 μs	1.27	0.09	False
Count - Duration of single invocation	54.29 μs	67.46 μs	1.24	0.16	False
Count - Duration of single invocation	54.68 μs	64.75 μs	1.18	0.17	False
Count - Duration of single invocation	75.52 μs	91.44 μs	1.21	0.09	False
Count - Duration of single invocation	65.67 μs	78.63 μs	1.20	0.12	False
Count - Duration of single invocation	53.92 μs	65.56 μs	1.22	0.20	False
Count - Duration of single invocation	74.77 μs	93.27 μs	1.25	0.11	False
Count - Duration of single invocation	59.81 μs	76.35 μs	1.28	0.18	False
Count - Duration of single invocation	54.37 μs	67.98 μs	1.25	0.17	False
Count - Duration of single invocation	52.74 μs	67.67 μs	1.28	0.14	False
Count - Duration of single invocation	101.14 μs	113.43 μs	1.12	0.04	False
Count - Duration of single invocation	73.78 μs	91.00 μs	1.23	0.14	False
Count - Duration of single invocation	64.92 μs	78.74 μs	1.21	0.09	False
Count - Duration of single invocation	66.68 μs	79.38 μs	1.19	0.12	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.14531909282701 > 89.34132659600334.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -39.28884482864162 (T) = (0 -101580.48373083991) / Math.Sqrt((1148431.9744116405 / (21)) + (6503800.564291497 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.20111755786329885 = (84571.64168971457 - 101580.48373083991) / 84571.64168971457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 137.5178669598681 > 128.88149175312986.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -36.97253116620927 (T) = (0 -137990.20209922327) / Math.Sqrt((890649.6256831465 / (21)) + (6592757.553125531 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.12702840523908515 = (122437.19985917334 - 137990.20209922327) / 122437.19985917334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.28374845297029 > 63.06970330459771.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -31.029156019966738 (T) = (0 -74466.6927743964) / Math.Sqrt((2592012.5569192297 / (21)) + (6058675.959281836 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.2579849578282081 = (59195.21716933412 - 74466.6927743964) / 59195.21716933412 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 182.10640463337833 > 170.26193179951682.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -15.309648825106004 (T) = (0 -178756.1582533575) / Math.Sqrt((34524622.96485283 / (21)) + (8951472.923447147 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.13076004032964444 = (158084.9622181959 - 178756.1582533575) / 158084.9622181959 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.7883603154836 > 67.57947517227338.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -27.605779192324366 (T) = (0 -77807.09849363472) / Math.Sqrt((2689697.4433716955 / (21)) + (5749161.277198183 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.2132433010009744 = (64131.488242663894 - 77807.09849363472) / 64131.488242663894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.11638846447671 > 81.77033112298922.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -15.92238843758559 (T) = (0 -90535.37888691686) / Math.Sqrt((7080045.505640174 / (21)) + (9089259.379738495 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.1444983385998028 = (79104.8582889856 - 90535.37888691686) / 79104.8582889856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.73122604752173 > 65.5028348233367.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -29.81453324872336 (T) = (0 -75226.99708549841) / Math.Sqrt((994495.6517385032 / (21)) + (7041163.112077407 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.2057778687062493 = (62388.76914054985 - 75226.99708549841) / 62388.76914054985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.63538352272727 > 63.53462708966572.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -39.16855257935032 (T) = (0 -75067.8893360268) / Math.Sqrt((1766845.292101261 / (21)) + (2811722.741186369 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.24298973323441747 = (60393.00834825932 - 75067.8893360268) / 60393.00834825932 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.82800665834374 > 97.87469580335508.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -18.34504067475396 (T) = (0 -106655.1817685864) / Math.Sqrt((5463546.624333933 / (20)) + (11720711.414193606 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.13965896745616022 = (93585.17312126458 - 106655.1817685864) / 93585.17312126458 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.72535721393038 > 141.5429148502845.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -20.132575655924427 (T) = (0 -150929.58388887384) / Math.Sqrt((13936389.742707904 / (20)) + (9555148.461810675 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.1437635305886894 = (131958.7308498909 - 150929.58388887384) / 131958.7308498909 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.36755053191492 > 56.17049628750478.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -28.80279370842201 (T) = (0 -68935.8610642737) / Math.Sqrt((1225793.1515690715 / (20)) + (10816317.05016138 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.2822920137647494 = (53759.87709841633 - 68935.8610642737) / 53759.87709841633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.83706451612903 > 77.96898535151698.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -43.13579031068089 (T) = (0 -92068.34303915765) / Math.Sqrt((1551526.4614469488 / (21)) + (4999157.367525814 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.24108700098626226 = (74183.63335204795 - 92068.34303915765) / 74183.63335204795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.46121724890828 > 56.273776243786706.
IsChangePoint: Marked as a change because one of 4/16/2022 2:55:52 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -35.73243920670328 (T) = (0 -69035.5414746832) / Math.Sqrt((1428183.6890469722 / (21)) + (5389829.27655387 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.2750459582002448 = (54143.571085177486 - 69035.5414746832) / 54143.571085177486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.74878829051382 > 57.65799448714719.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -31.870451297016544 (T) = (0 -68540.25758823815) / Math.Sqrt((1117889.750538502 / (21)) + (7389207.457263924 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.2633877177144461 = (54251.16662700514 - 68540.25758823815) / 54251.16662700514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.44465600775196 > 78.51058933184576.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -46.00424389140184 (T) = (0 -91302.95911947236) / Math.Sqrt((521299.62743420823 / (20)) + (4822568.602875332 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.21412865203765943 = (75200.39904028256 - 91302.95911947236) / 75200.39904028256 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.62613100848257 > 69.99382469506435.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -27.906426037526394 (T) = (0 -81484.59460883355) / Math.Sqrt((2686148.2238005637 / (21)) + (7177348.192191951 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.21718951971640846 = (66944.870366464 - 81484.59460883355) / 66944.870366464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 65.5564945652174 > 56.09077357653505.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -36.12834608440349 (T) = (0 -69505.65096379076) / Math.Sqrt((1482913.2344738361 / (21)) + (6180276.145040504 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.2947330859762286 = (53683.38209367957 - 69505.65096379076) / 53683.38209367957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 93.2706268939394 > 78.61913677536234.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -46.940394701236436 (T) = (0 -91237.1899184994) / Math.Sqrt((1112625.7471722693 / (20)) + (3258488.1931026806 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.2163515908831021 = (75008.89594945067 - 91237.1899184994) / 75008.89594945067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.34721213054189 > 62.87895375844595.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -33.85642924330733 (T) = (0 -73320.92913505333) / Math.Sqrt((1315013.61188243 / (21)) + (4543786.753055526 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.2208660321581808 = (60056.490395953246 - 73320.92913505333) / 60056.490395953246 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.97952823315119 > 56.92774937020185.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -31.157370016448226 (T) = (0 -69038.78428497002) / Math.Sqrt((749623.2100224892 / (21)) + (10211345.793338707 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.2807446520823269 = (53905.19036930726 - 69038.78428497002) / 53905.19036930726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.6690229257642 > 57.49758836976543.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -32.031729690541326 (T) = (0 -69064.76990108256) / Math.Sqrt((1394326.9957983496 / (21)) + (6844419.41284537 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.26439956192710745 = (54622.58290869618 - 69064.76990108256) / 54622.58290869618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.43335535307517 > 105.39529072612208.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -27.446009866565646 (T) = (0 -115809.90179090401) / Math.Sqrt((2677807.3715454815 / (21)) + (9808338.385373466 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.1558446358630086 = (100195.04196118437 - 115809.90179090401) / 100195.04196118437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.99605154388524 > 77.195971760551.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.434277701939543 (T) = (0 -85337.36526514705) / Math.Sqrt((7045068.929182456 / (20)) + (12277550.555882877 / (49))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (49) - 2, .025) and -0.1885231320225898 = (71801.18162271079 - 85337.36526514705) / 71801.18162271079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.73526725113123 > 67.61676726583855.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -30.075634404299397 (T) = (0 -77222.93219050714) / Math.Sqrt((1282462.6920450428 / (21)) + (6546235.866141087 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.2057672416698977 = (64044.64271525501 - 77222.93219050714) / 64044.64271525501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.37544056251373 > 69.51882826323852.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -29.09749564081086 (T) = (0 -79836.87071235188) / Math.Sqrt((2725580.1504175435 / (21)) + (4963946.048046676 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.21242275580462905 = (65849.03683976785 - 79836.87071235188) / 65849.03683976785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetBytes - Duration of single invocation	27.98 ns	30.06 ns	1.07	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.063817232871795 > 29.474937282048018.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.461175950784018 (T) = (0 -29.86454243002928) / Math.Sqrt((0.05622276188060895 / (21)) + (0.3620379016191968 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.0623153888074591 = (28.11268926787817 - 29.86454243002928) / 28.11268926787817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2022-06-07T12:20:49Z

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	38.59 μs	42.80 μs	1.11	0.00	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.79841757741349 > 40.44708071265292.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -120.23458824212399 (T) = (0 -43059.47063357147) / Math.Sqrt((16270.560797385531 / (21)) + (31466.169347696905 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.11685833633880044 = (38554.10237141242 - 43059.47063357147) / 38554.10237141242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	14.61 μs	17.75 μs	1.21	0.16	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.745394328442437 > 15.667288794279344.
IsChangePoint: Marked as a change because one of 4/21/2022 7:46:01 AM, 5/9/2022 5:23:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -4.071437441872856 (T) = (0 -16776.386824311085) / Math.Sqrt((866338.0602542905 / (21)) + (1031291.9076750234 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.06402546737333263 = (15766.90346117889 - 16776.386824311085) / 15766.90346117889 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOf_Word_NotFound - Duration of single invocation	24.80 ns	30.43 ns	1.23	0.06	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.433843499697037 > 25.87287708005478.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -38.28460583743384 (T) = (0 -30.234897100780387) / Math.Sqrt((0.20291577514232229 / (21)) + (0.40321468896107365 / (48))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (48) - 2, .025) and -0.20508155873082093 = (25.089502765790773 - 30.234897100780387) / 25.089502765790773 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IList - Duration of single invocation	2.10 μs	2.26 μs	1.08	0.02	False
SortedList - Duration of single invocation	35.36 μs	38.74 μs	1.10	0.00	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2553911706062237 > 2.2001298433723764.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/23/2022 3:43:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -169.0388128230885 (T) = (0 -2256.1128118110146) / Math.Sqrt((0.43308028464838483 / (20)) + (42.53103408580171 / (48))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (48) - 2, .025) and -0.07687139290000758 = (2095.0624435619166 - 2256.1128118110146) / 2095.0624435619166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSet&lt;Int32&gt;.SortedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.742540936785986 > 37.13587016898202.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -96.38465918377705 (T) = (0 -38916.31412533478) / Math.Sqrt((9851.362018101976 / (21)) + (46713.04461476093 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.10303439789243128 = (35281.14281811357 - 38916.31412533478) / 35281.14281811357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2022-06-07T12:20:49Z

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	12.70 ns	14.78 ns	1.16	0.20	False
TryParse - Duration of single invocation	13.09 ns	14.12 ns	1.08	0.18	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.Parse(value: "32767")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.778821856811254 > 13.335358331549736.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -18.667874077332495 (T) = (0 -14.433117986486662) / Math.Sqrt((0.11398696517710356 / (21)) + (0.11377924663560174 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.12849443063260077 = (12.789711313325562 - 14.433117986486662) / 12.789711313325562 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int16.TryParse(value: "-32768")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.122244665081979 > 13.74322045685181.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.74412788332246 (T) = (0 -15.439452099256673) / Math.Sqrt((0.11825316450030281 / (21)) + (0.4760764815886171 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.16478275280888535 = (13.255220393695115 - 15.439452099256673) / 13.255220393695115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	1.98 ms	2.56 ms	1.29	0.53	False
Count - Duration of single invocation	2.22 ms	2.60 ms	1.17	0.34	False
Count - Duration of single invocation	2.29 ms	2.69 ms	1.17	0.40	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.555546681415929 > 2.108958592246588.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -14.565327536229416 (T) = (0 -2535519.1797791906) / Math.Sqrt((11642453519.44851 / (21)) + (15268473907.993471 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.20170983589488478 = (2109926.293389327 - 2535519.1797791906) / 2109926.293389327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5983969375 > 2.2936269270833334.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -22.652539126674125 (T) = (0 -2576286.200456584) / Math.Sqrt((2290241225.6038823 / (21)) + (9098339013.834991 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.17519411894280795 = (2192221.828657706 - 2576286.200456584) / 2192221.828657706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6851410486891387 > 2.40014402881356.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -18.978339928304013 (T) = (0 -2639174.1637789314) / Math.Sqrt((2868751939.050661 / (19)) + (10759190854.333109 / (51))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (51) - 2, .025) and -0.15849243813012573 = (2278110.824822225 - 2639174.1637789314) / 2278110.824822225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	25.56 μs	27.83 μs	1.09	0.00	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.828146929824563 > 26.80421557762019.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -31.647439514948726 (T) = (0 -27346.234373650957) / Math.Sqrt((35140.43039323096 / (21)) + (94180.69743059682 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.07413804167206615 = (25458.77095189945 - 27346.234373650957) / 25458.77095189945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableDictionary - Duration of single invocation	37.15 μs	41.10 μs	1.11	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.101531058617674 > 38.71304640101302.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -35.11459706129154 (T) = (0 -40883.95645512673) / Math.Sqrt((153664.46472215126 / (21)) + (391656.82019197266 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.1175439091015285 = (36583.758474417526 - 40883.95645512673) / 36583.758474417526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ImmutableArray - Duration of single invocation	1.12 ms	1.20 ms	1.07	0.03	True
Queue - Duration of single invocation	1.12 ms	1.20 ms	1.07	0.00	True
Stack - Duration of single invocation	1.12 ms	1.20 ms	1.07	0.03	False
List - Duration of single invocation	1.11 ms	1.19 ms	1.08	0.01	True
Span - Duration of single invocation	893.19 μs	967.51 μs	1.08	0.02	False
ICollection - Duration of single invocation	1.11 ms	1.20 ms	1.08	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.201222829861111 > 1.1742438807091349.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -64.98996034047858 (T) = (0 -1198879.4529533132) / Math.Sqrt((16708999.624230709 / (21)) + (32444069.427949704 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.06936521881685985 = (1121113.1911319757 - 1198879.4529533132) / 1121113.1911319757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2011723557692304 > 1.1775175823317305.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 5/11/2022 7:02:26 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -77.68402628871813 (T) = (0 -1200767.8326440882) / Math.Sqrt((18310634.04558377 / (20)) + (12700000.655431796 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.07522795020684411 = (1116756.5281511643 - 1200767.8326440882) / 1116756.5281511643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.198246785714286 > 1.1733238874699519.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -99.22496896853453 (T) = (0 -1200667.1203774498) / Math.Sqrt((6477876.233319225 / (21)) + (19527439.692244142 / (49))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (49) - 2, .025) and -0.0746762459595308 = (1117236.1210100325 - 1200667.1203774498) / 1117236.1210100325 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.194428660714286 > 1.16447954375.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -89.7201098757776 (T) = (0 -1193242.5458123332) / Math.Sqrt((5202384.524441343 / (21)) + (32331946.750240497 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.07596579084156849 = (1108996.7320234564 - 1193242.5458123332) / 1108996.7320234564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 967.5052941176468 > 937.5439835069445.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 4/22/2022 6:17:37 AM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -334.29557810954356 (T) = (0 -967527.7658505711) / Math.Sqrt((1736022.343771344 / (41)) + (182542.17050032664 / (30))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (30) - 2, .025) and -0.08229131832109686 = (893962.4197960374 - 967527.7658505711) / 893962.4197960374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.196064392361111 > 1.1634300024038464.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 5/11/2022 7:02:26 PM, 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -63.201986630655405 (T) = (0 -1194415.4476816363) / Math.Sqrt((26538140.003756035 / (20)) + (20847137.503429342 / (49))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (49) - 2, .025) and -0.07532265670885334 = (1110750.7502327533 - 1194415.4476816363) / 1110750.7502327533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	32.12 μs	33.76 μs	1.05	0.12	False
Span - Duration of single invocation	27.46 μs	32.76 μs	1.19	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.76229707792208 > 33.757713295687886.
IsChangePoint: Marked as a change because one of 4/3/2022 8:21:23 AM, 5/16/2022 8:28:13 AM, 5/20/2022 9:09:19 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -28.663328720853137 (T) = (0 -35811.724909659875) / Math.Sqrt((17414.580162452392 / (20)) + (774767.8235075272 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.11289040765714044 = (32179.02199826738 - 35811.724909659875) / 32179.02199826738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.75943784435262 > 28.866759367617586.
IsChangePoint: Marked as a change because one of 4/5/2022 2:02:34 AM, 5/31/2022 9:21:33 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -84.98649346693126 (T) = (0 -32912.06068374717) / Math.Sqrt((528.637296108339 / (30)) + (170527.22381743268 / (42))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (42) - 2, .025) and -0.19745418472132745 = (27485.027071333418 - 32912.06068374717) / 27485.027071333418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AddOperatorBenchmark - Duration of single invocation	11.77 ns	13.31 ns	1.13	0.28	False
AddBenchmark - Duration of single invocation	13.02 ns	15.23 ns	1.17	0.42	False
CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation	7.99 ns	9.22 ns	1.15	0.06	False
LerpBenchmark - Duration of single invocation	11.11 ns	12.99 ns	1.17	0.45	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.310602819757564 > 12.35068310926106.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -19.095127109095415 (T) = (0 -13.30847942135133) / Math.Sqrt((0.095984199647061 / (21)) + (0.006385931931208392 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.10907627147191414 = (11.999607027647377 - 13.30847942135133) / 11.999607027647377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.234574928787104 > 13.68541653008154.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -12.292357395491345 (T) = (0 -15.22752327992928) / Math.Sqrt((0.3636769141778483 / (21)) + (0.002039934766780247 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.11901478545266057 = (13.60797326173799 - 15.22752327992928) / 13.60797326173799 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.222406598885863 > 8.53413405389611.
IsChangePoint: Marked as a change because one of 5/16/2022 12:27:40 PM, 5/31/2022 2:22:18 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -9.643849509019539 (T) = (0 -9.002429335726461) / Math.Sqrt((0.08276347369870621 / (27)) + (0.07408724374435564 / (44))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (44) - 2, .025) and -0.07970828659048707 = (8.337834809209825 - 9.002429335726461) / 8.337834809209825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.9865629905953 > 11.674874582751029.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -16.94204016247189 (T) = (0 -12.985296889621816) / Math.Sqrt((0.18461463988239368 / (21)) + (0.0013864780225862619 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.13962738039261977 = (11.394335651314533 - 12.985296889621816) / 11.394335651314533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	b8acc97aba7662204c2a6de8f6090fc115a41c9f
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ImmutableArray - Duration of single invocation	567.63 μs	608.29 μs	1.07	0.01	True
ImmutableHashSet - Duration of single invocation	62.46 μs	69.28 μs	1.11	0.01	False
Stack - Duration of single invocation	562.55 μs	602.98 μs	1.07	0.01	False
ICollection - Duration of single invocation	556.93 μs	602.43 μs	1.08	0.01	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 608.2949919871794 > 595.3304393458104.
IsChangePoint: Marked as a change because one of 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -50.8529824259434 (T) = (0 -609286.8816868864) / Math.Sqrt((10983179.225586748 / (20)) + (3227162.0425035483 / (49))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (49) - 2, .025) and -0.07003877265867861 = (569406.3591481063 - 609286.8816868864) / 569406.3591481063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ImmutableHashSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.2792662241888 > 65.83298882128514.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -24.528669162801208 (T) = (0 -69291.47062723782) / Math.Sqrt((1935527.2160877325 / (42)) + (34846.747108892734 / (29))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (29) - 2, .025) and -0.08339613215294264 = (63957.649995981315 - 69291.47062723782) / 63957.649995981315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 602.9823832417584 > 589.94089453125.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -71.80689866005628 (T) = (0 -602024.652174028) / Math.Sqrt((3053338.604634835 / (21)) + (9751759.017203404 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.07479876089564666 = (560127.7877100932 - 602024.652174028) / 560127.7877100932 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 602.4335737179487 > 585.3266105872846.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -118.82583215455931 (T) = (0 -600334.457868359) / Math.Sqrt((964695.5522760156 / (20)) + (3984976.511513186 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.07619039334437669 = (557832.9462714823 - 600334.457868359) / 557832.9462714823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2022-06-07T12:20:49Z

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Min - Duration of single invocation	6.26 μs	7.40 μs	1.18	0.01	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Min

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.401327725652409 > 6.57669710732106.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -80.49779127821778 (T) = (0 -7398.574809549225) / Math.Sqrt((51.99463268224369 / (20)) + (9792.423116412796 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.18101515710003602 = (6264.589209605327 - 7398.574809549225) / 6264.589209605327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	1.65 μs	1.83 μs	1.11	0.05	True
Stack - Duration of single invocation	1.61 μs	1.78 μs	1.10	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8320642313249411 > 1.73302019686531.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -171.80274209076939 (T) = (0 -1833.9035203221283) / Math.Sqrt((0.31880168515008894 / (21)) + (57.39098658620936 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.1111680390086233 = (1650.4286084023122 - 1833.9035203221283) / 1650.4286084023122 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7799645665515922 > 1.6949704092689348.
IsChangePoint: Marked as a change because one of 4/3/2022 1:52:36 PM, 4/21/2022 3:06:05 PM, 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -26.8587787178787 (T) = (0 -1781.2000715214133) / Math.Sqrt((671.3237184412767 / (19)) + (17.192874197082986 / (51))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (51) - 2, .025) and -0.09897176025176592 = (1620.7878454614286 - 1781.2000715214133) / 1620.7878454614286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	11.04 ns	13.39 ns	1.21	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.389415708028562 > 11.647724902625825.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -35.3821005451139 (T) = (0 -13.809225814719719) / Math.Sqrt((0.0038501661426411475 / (21)) + (0.28952613718363696 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.24400395643086178 = (11.100628533641803 - 13.809225814719719) / 11.100628533641803 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.TryAddDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	10.47 μs	11.22 μs	1.07	0.30	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<Int32>.Dictionary(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.21862364620939 > 10.954401388037457.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -7.327712084905677 (T) = (0 -10934.838466546367) / Math.Sqrt((18584.42751013444 / (21)) + (268326.80687956774 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.05544978493124501 = (10360.35879931388 - 10934.838466546367) / 10360.35879931388 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringHex - Duration of single invocation	21.31 ns	23.18 ns	1.09	0.02	False
Parse - Duration of single invocation	18.91 ns	21.75 ns	1.15	0.22	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToStringHex(value: -2147483648)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.178391418246896 > 22.353339204547403.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -48.86652128089794 (T) = (0 -23.215633176983413) / Math.Sqrt((0.010692965806884766 / (21)) + (0.05248145449433673 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.08998339971780596 = (21.299070410607982 - 23.215633176983413) / 21.299070410607982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.Parse(value: "2147483647")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.752073641604966 > 19.873683561163777.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -14.58330924379385 (T) = (0 -23.157740282854903) / Math.Sqrt((0.9395384739492012 / (20)) + (1.158482389968456 / (48))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (48) - 2, .025) and -0.20182362864482645 = (19.268834237322757 - 23.157740282854903) / 19.268834237322757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	38.48 μs	42.47 μs	1.10	0.00	True
SortedDictionary - Duration of single invocation	56.81 μs	60.14 μs	1.06	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.474327445652186 > 40.33973975966692.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -150.83202626427587 (T) = (0 -42488.232892924985) / Math.Sqrt((4370.218651098903 / (21)) + (25450.987255115757 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.10505222385190822 = (38449.0723387015 - 42488.232892924985) / 38449.0723387015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.13718550446998 > 59.70867556818182.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/23/2022 3:43:39 PM, 6/6/2022 10:51:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -76.29260544540821 (T) = (0 -60095.22985312899) / Math.Sqrt((43632.48676773059 / (66)) + (7086.29501608512 / (5))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (66) + (5) - 2, .025) and -0.0614320108554852 = (56617.125956747695 - 60095.22985312899) / 56617.125956747695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	620.81 ms	772.24 ms	1.24	0.15	False
Count - Duration of single invocation	614.60 ms	753.52 ms	1.23	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 772.2439 > 652.8690630000001.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -28.313663336023282 (T) = (0 -758404286.4612513) / Math.Sqrt((186590418318822.28 / (20)) + (648590280054667.5 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.21255496111124672 = (625459720 - 758404286.4612513) / 625459720 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 753.5163666666666 > 640.1903025.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -43.27418172223156 (T) = (0 -739055580.9954752) / Math.Sqrt((80135949142868.62 / (21)) + (241459569420188.25 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.20658880517361775 = (612516524.1269841 - 739055580.9954752) / 612516524.1269841 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Enumerator - Duration of single invocation	186.67 ns	199.28 ns	1.07	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.Enumerator

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 199.27866926582413 > 196.10605474516325.
IsChangePoint: Marked as a change because one of 3/30/2022 12:01:24 PM, 4/1/2022 10:26:17 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -28.93759259779189 (T) = (0 -200.2400874872708) / Math.Sqrt((1.8214296465912974 / (20)) + (4.07761922476578 / (48))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (48) - 2, .025) and -0.0645443864763635 = (188.09933153662524 - 200.2400874872708) / 188.09933153662524 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in LinqBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Where00ForX - Duration of single invocation	257.78 ms	282.39 ms	1.10	0.15	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Where00ForX

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 282.3931 > 271.13687250000004.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -12.08722555353121 (T) = (0 -282890738.6568985) / Math.Sqrt((61337870998247.79 / (21)) + (8341065075148.621 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.08112793550076365 = (261662592.712367 - 282890738.6568985) / 261662592.712367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	35.58 μs	39.82 μs	1.12	0.00	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.81652954931973 > 37.368200049942146.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -109.8145634641523 (T) = (0 -40148.81260594465) / Math.Sqrt((13955.690791560355 / (21)) + (52688.81532332001 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.12788038811141214 = (35596.693611431736 - 40148.81260594465) / 35596.693611431736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Byte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	7.99 ns	9.88 ns	1.24	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 255)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.884576888080073 > 8.375008099148829.
IsChangePoint: Marked as a change because one of 4/19/2022 11:38:14 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -42.413548176456615 (T) = (0 -9.647180216898889) / Math.Sqrt((0.005190478984685617 / (20)) + (0.06908205659452521 / (50))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (50) - 2, .025) and -0.21670300197589729 = (7.928952424077275 - 9.647180216898889) / 7.928952424077275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
InequalityOperatorBenchmark - Duration of single invocation	7.10 ns	9.02 ns	1.27	0.24	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix3x2.InequalityOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.015348023441021 > 7.479110318910252.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -16.27936411115159 (T) = (0 -8.23828031383183) / Math.Sqrt((0.01511470328332411 / (21)) + (0.170823015983009 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.14558989626022417 = (7.1912997318898135 - 8.23828031383183) / 7.1912997318898135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.TryAddDefaultSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	24.84 μs	27.01 μs	1.09	0.01	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<String>.Dictionary(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.0142559009787 > 26.08942236326315.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -18.88976165564499 (T) = (0 -26641.017422680758) / Math.Sqrt((32840.27102441176 / (19)) + (378218.355274711 / (51))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (51) - 2, .025) and -0.07273575848879309 = (24834.65029655677 - 26641.017422680758) / 24834.65029655677 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler bot assigned AndyAyersMS Jun 7, 2022

performanceautofiler bot added CoreClr untriaged labels Jun 7, 2022

AndyAyersMS mentioned this issue Jun 10, 2022

JIT: defer some flow graph reordering until after loop recognition dotnet/runtime#69878

Merged

AndyAyersMS removed the untriaged label Jun 10, 2022

AndyAyersMS closed this as completed Jun 10, 2022

mrsharm mentioned this issue Aug 12, 2022

.NET 7.0 Preview 7 Microbenchmarks Performance Study Report dotnet/runtime#73866

Closed

24 tasks

performanceautofiler bot added perf-regression arch-x86 os-windows branch-refs/heads/main runtime-coreclr kind-micro and removed refs/heads/main labels Mar 24, 2023

[Perf] Changes at 5/30/2022 5:18:40 PM #5818

[Perf] Changes at 5/30/2022 5:18:40 PM #5818

Comments

performanceautofiler bot commented Jun 7, 2022 • edited Loading

Run Information

Regressions in System.Tests.Perf_Double

Repro

Payloads

Histogram

System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in PerfLabTests.CastingPerf

Repro

Payloads

Histogram

PerfLabTests.CastingPerf.IntObj

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Linq.Tests.Perf_Enumerable

Repro

Payloads

Histogram

System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

performanceautofiler bot commented Jun 7, 2022 • edited Loading

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Text.Tests.Perf_Encoding

Repro

Payloads

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")

Description of detection logic

Docs

performanceautofiler bot commented Jun 7, 2022 • edited Loading

Run Information

performanceautofiler bot commented Jun 7, 2022 •

edited

Loading

performanceautofiler bot commented Jun 7, 2022 •

edited

Loading

performanceautofiler bot commented Jun 7, 2022 •

edited

Loading

performanceautofiler bot commented Jun 7, 2022 •

edited

Loading