[Perf] Regressions in IndexOf #70158

performanceautofiler · 2022-06-02T12:12:41Z

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	170.35 μs	202.84 μs	1.19	0.01	False
Count - Duration of single invocation	14.00 ms	22.13 ms	1.58	0.07	False
Count - Duration of single invocation	11.47 ms	18.84 ms	1.64	0.05	False
Count - Duration of single invocation	359.33 μs	401.71 μs	1.12	0.01	False
Count - Duration of single invocation	136.13 μs	156.40 μs	1.15	0.01	False
Count - Duration of single invocation	13.95 ms	22.01 ms	1.58	0.08	False
Count - Duration of single invocation	115.78 μs	148.59 μs	1.28	0.01	False
Count - Duration of single invocation	214.35 μs	225.78 μs	1.05	0.00	False
Count - Duration of single invocation	85.88 μs	117.71 μs	1.37	0.01	False
Count - Duration of single invocation	90.20 μs	123.32 μs	1.37	0.01	False
Count - Duration of single invocation	141.57 μs	177.60 μs	1.25	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 202.84127705627705 > 178.70211256270906.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 3/17/2022 6:54:53 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -177.8589860518688 (T) = (0 -202849.65972862608) / Math.Sqrt((155692.5460968798 / (33)) + (149428.62237488898 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.19489056074647046 = (169764.21639978653 - 202849.65972862608) / 169764.21639978653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.12947 > 14.690156749999998.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/19/2022 10:22:09 AM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -212.91215108523403 (T) = (0 -22190245.87708588) / Math.Sqrt((12541864947.533058 / (20)) + (16473521910.62474 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.604644335031837 = (13828762.79349817 - 22190245.87708588) / 13828762.79349817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.840791666666664 > 12.045469038461539.
IsChangePoint: Marked as a change because one of 4/19/2022 10:22:09 AM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -132.64493146334232 (T) = (0 -18908716.665649164) / Math.Sqrt((13902977425.303988 / (20)) + (44054699498.343704 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.6481416154247686 = (11472749.96801528 - 18908716.665649164) / 11472749.96801528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 401.70645264847514 > 376.95830819037076.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -20.164200721246957 (T) = (0 -393637.95899411716) / Math.Sqrt((14259805.299575618 / (20)) + (44260431.42444706 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.10039004541073393 = (357725.84515446704 - 393637.95899411716) / 357725.84515446704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.39642633228843 > 142.97913152173916.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/19/2022 10:22:09 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -23.16142001782066 (T) = (0 -157302.23190307943) / Math.Sqrt((33030007.745005228 / (33)) + (1462936.1060213188 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.20114451640660536 = (130960.28808729147 - 157302.23190307943) / 130960.28808729147 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.00644666666667 > 14.614975180555557.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/19/2022 10:22:09 AM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -239.19249976789942 (T) = (0 -22111815.36782662) / Math.Sqrt((9111932007.963709 / (20)) + (13517419130.350264 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.6019209966703583 = (13803312.03210814 - 22111815.36782662) / 13803312.03210814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.59220238095236 > 121.26854693244485.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -152.93599467207557 (T) = (0 -148167.83849206349) / Math.Sqrt((91989.7227325268 / (33)) + (215972.8188658445 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.2842495087101177 = (115373.09338033634 - 148167.83849206349) / 115373.09338033634 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 225.7829285714286 > 225.3496136558219.
IsChangePoint: Marked as a change because one of 2/15/2022 12:48:49 AM, 2/28/2022 3:30:29 AM, 4/16/2022 8:46:33 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -84.39154468580743 (T) = (0 -225545.31332815735) / Math.Sqrt((320327.3906533836 / (33)) + (33569.01611444854 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.05036162409451314 = (214731.10608224434 - 225545.31332815735) / 214731.10608224434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.70799185463659 > 90.13683205212648.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -476.7137241354218 (T) = (0 -117618.80106860558) / Math.Sqrt((54350.887999398336 / (33)) + (14598.745742718116 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3772115474517505 = (85403.58326666315 - 117618.80106860558) / 85403.58326666315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.31891404199474 > 94.92197886560695.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -251.56982118225088 (T) = (0 -123098.91966629171) / Math.Sqrt((88242.72862351728 / (33)) + (73013.34592780822 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3672739493667976 = (90032.3740705368 - 123098.91966629171) / 90032.3740705368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 177.59994297933002 > 148.42813572464257.
IsChangePoint: Marked as a change because one of 2/12/2022 1:03:18 PM, 2/18/2022 6:05:18 PM, 4/19/2022 10:22:09 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -88.84000320256142 (T) = (0 -177886.44049185622) / Math.Sqrt((4101888.2189159435 / (33)) + (156271.74844279446 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.24528786142125564 = (142847.64671907522 - 177886.44049185622) / 142847.64671907522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfAnyThreeValues - Duration of single invocation	25.12 ns	36.48 ns	1.45	0.21	False
LastIndexOfAnyValues - Duration of single invocation	25.55 ns	35.12 ns	1.37	0.22	False
IndexOfAnyTwoValues - Duration of single invocation	20.93 ns	34.46 ns	1.65	0.20	False
LastIndexOfValue - Duration of single invocation	18.41 ns	25.38 ns	1.38	0.24	False
IndexOfValue - Duration of single invocation	16.19 ns	23.48 ns	1.45	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.48333542747834 > 26.578851645102183.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -95.6372254266481 (T) = (0 -36.39163999209435) / Math.Sqrt((0.360819912230195 / (33)) + (0.014631029646441149 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.44799960970261943 = (25.132354835073624 - 36.39163999209435) / 25.132354835073624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.117858079786885 > 26.92373391220861.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -56.523029703634535 (T) = (0 -35.26292345204584) / Math.Sqrt((0.7999462918984083 / (33)) + (0.02373966184242386 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3753482378502403 = (25.63926900954489 - 35.26292345204584) / 25.63926900954489 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.46131285641188 > 21.89515796744658.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -133.9092095156149 (T) = (0 -34.3675363500865) / Math.Sqrt((0.07690242307970953 / (33)) + (0.03900784982898168 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.6452802487516671 = (20.888560703358824 - 34.3675363500865) / 20.888560703358824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.38491067501782 > 19.32052285948416.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -90.23805183039804 (T) = (0 -25.336819571786826) / Math.Sqrt((0.14746862069050773 / (33)) + (0.007930948957717818 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.38338540445275826 = (18.31508377219695 - 25.336819571786826) / 18.31508377219695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.47667844509549 > 16.999061106314183.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 4/14/2022 10:59:17 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -92.95898376011084 (T) = (0 -23.37895094740987) / Math.Sqrt((0.12489434891207174 / (33)) + (0.011613701574718261 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.45082830320150435 = (16.114209307758994 - 23.37895094740987) / 16.114209307758994 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Utf8JsonReaderCommentParsing - Duration of single invocation	284.28 ns	367.26 ns	1.29	0.00	False
Utf8JsonReaderCommentParsing - Duration of single invocation	778.17 ns	853.28 ns	1.10	0.09	False
Utf8JsonReaderCommentParsing - Duration of single invocation	282.10 ns	367.97 ns	1.30	0.00	False
Utf8JsonReaderCommentParsing - Duration of single invocation	818.34 ns	899.92 ns	1.10	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 367.2607359541503 > 295.7002994826678.
IsChangePoint: Marked as a change because one of 4/12/2022 9:30:02 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -89.11344484490607 (T) = (0 -368.1217465132745) / Math.Sqrt((8.516265264479829 / (33)) + (3.6007944707960386 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.31479682896798505 = (279.9837498864535 - 368.1217465132745) / 279.9837498864535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 853.2784780578896 > 816.3021910778831.
IsChangePoint: Marked as a change because one of 4/12/2022 9:30:02 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -33.80329864409164 (T) = (0 -857.5544626935844) / Math.Sqrt((97.21382851091069 / (33)) + (29.291008658107828 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.13245218249484803 = (757.2544571412716 - 857.5544626935844) / 757.2544571412716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 367.96651018691074 > 294.9228501471663.
IsChangePoint: Marked as a change because one of 4/12/2022 9:30:02 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -172.2474661442676 (T) = (0 -366.2352230020873) / Math.Sqrt((1.9874780311813227 / (33)) + (0.9557291908109937 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.30856518870985544 = (279.8754132861864 - 366.2352230020873) / 279.8754132861864 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 899.9235548096036 > 857.5916561268206.
IsChangePoint: Marked as a change because one of 3/21/2022 10:36:51 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -26.160620230156795 (T) = (0 -896.8045289205475) / Math.Sqrt((107.50742051535703 / (33)) + (55.74403807983959 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.12450684236068399 = (797.5091792575315 - 896.8045289205475) / 797.5091792575315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	926.23 μs	1.22 ms	1.32	0.01	False
Count - Duration of single invocation	849.58 μs	1.15 ms	1.35	0.01	True
Count - Duration of single invocation	929.96 μs	1.23 ms	1.32	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w.+-]+@[\w.-]+.[\w.-]+", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2235153846153846 > 979.5645753676471.
IsChangePoint: Marked as a change because one of 3/16/2022 7:30:14 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -142.04426056061916 (T) = (0 -1222350.4863200902) / Math.Sqrt((66460442.677282654 / (33)) + (10514080.77879062 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.30852300283045225 = (934145.2031611495 - 1222350.4863200902) / 934145.2031611495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1485574617346936 > 890.3775032894736.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -157.51298595488143 (T) = (0 -1147964.944302721) / Math.Sqrt((98927024.773689 / (33)) + (1595280.1406405908 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3331392660864016 = (861099.0415672901 - 1147964.944302721) / 861099.0415672901 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2314026275704488 > 983.2405719235186.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -80.77230352147373 (T) = (0 -1232654.619637331) / Math.Sqrt((238218922.30535525 / (33)) + (27360401.702412676 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.30452099705913704 = (944909.7580002017 - 1232654.619637331) / 944909.7580002017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2022-06-02T12:12:44Z

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfValue - Duration of single invocation	30.31 ns	44.21 ns	1.46	0.05	False
LastIndexOfValue - Duration of single invocation	32.50 ns	42.37 ns	1.30	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.20662707047519 > 31.814927896471897.
IsChangePoint: Marked as a change because one of 4/12/2022 9:30:02 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -39.86077735344293 (T) = (0 -43.80517214744365) / Math.Sqrt((0.34946877507288165 / (33)) + (0.5354665807215608 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.4538272321664689 = (30.130933840168833 - 43.80517214744365) / 30.130933840168833 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.37021381770035 > 33.96682687723006.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -30.239506565862296 (T) = (0 -42.85029849297587) / Math.Sqrt((0.08332713167146938 / (33)) + (0.5732627898144075 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.318513428736879 = (32.49894734407519 - 42.85029849297587) / 32.49894734407519 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Tests.Perf_Array

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfChar - Duration of single invocation	14.97 ns	18.34 ns	1.23	0.26	False
IndexOfShort - Duration of single invocation	17.31 ns	21.04 ns	1.22	0.13	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfChar

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.343541001603377 > 15.69471470289251.
IsChangePoint: Marked as a change because one of 1/27/2022 5:13:24 AM, 1/27/2022 11:49:03 PM, 4/13/2022 4:27:35 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -48.135586134685624 (T) = (0 -18.350786189390078) / Math.Sqrt((0.17310568150005867 / (33)) + (0.00027591764911557654 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.2360599112244644 = (14.84619476996988 - 18.350786189390078) / 14.84619476996988 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfShort

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.038919975702186 > 18.02993616914522.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 4/14/2022 10:59:17 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -13.381860095155647 (T) = (0 -20.58246913694932) / Math.Sqrt((0.16895745685038036 / (33)) + (0.27251956783791104 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.18871515105857992 = (17.314887522565964 - 20.58246913694932) / 17.314887522565964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfString - Duration of single invocation	31.75 ns	41.30 ns	1.30	0.14	False
IndexOfString - Duration of single invocation	65.73 ns	92.69 ns	1.41	0.04	False
IndexOfString - Duration of single invocation	65.74 ns	92.72 ns	1.41	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.29666065641675 > 33.33409307609369.
IsChangePoint: Marked as a change because one of 2/4/2022 3:19:13 PM, 3/22/2022 12:26:23 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -464.4601427213283 (T) = (0 -41.299543377609524) / Math.Sqrt((0.012079487490324885 / (33)) + (0.0002990311532526343 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.30221397657743093 = (31.71486723415137 - 41.299543377609524) / 31.71486723415137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.68599524909509 > 68.9692279829118.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -876.7787427693668 (T) = (0 -92.65366134277836) / Math.Sqrt((0.021308457533832002 / (33)) + (0.0015167204854237158 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.4114787949016997 = (65.6429708171642 - 92.65366134277836) / 65.6429708171642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.72117049498031 > 68.98537950551021.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -863.4428713039174 (T) = (0 -92.67803727087338) / Math.Sqrt((0.02215716909440243 / (33)) + (0.001542537178997674 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.41172323609715583 = (65.64887146512562 - 92.67803727087338) / 65.64887146512562 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	5.06 ms	5.39 ms	1.06	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.390474583333333 > 5.313420700420673.
IsChangePoint: Marked as a change because one of 2/11/2022 12:34:36 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -190.7233516916739 (T) = (0 -5393141.875) / Math.Sqrt((70926056.04735205 / (33)) + (4176455.0733030327 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.0650699877404016 = (5063650.217430139 - 5393141.875) / 5063650.217430139 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetByteCount - Duration of single invocation	16.36 μs	26.41 μs	1.61	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.409548082346312 > 17.176569587067565.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -121.57786131628713 (T) = (0 -26475.546662324614) / Math.Sqrt((26.762740483768454 / (33)) + (34609.80562222117 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.6183215858608773 = (16359.879824652258 - 26475.546662324614) / 16359.879824652258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.IO.Tests.Perf_Directory

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetCurrentDirectory - Duration of single invocation	135.84 ns	146.36 ns	1.08	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Directory*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Directory.GetCurrentDirectory

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 146.35789097543497 > 140.3632342261326.
IsChangePoint: Marked as a change because one of 1/25/2022 5:52:36 PM, 3/1/2022 1:15:47 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -9.806035740512998 (T) = (0 -145.71765263733852) / Math.Sqrt((3.752785491640121 / (33)) + (3.6714801819520533 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.0660643158856707 = (136.68748729881125 - 145.71765263733852) / 136.68748729881125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

dotnet-issue-labeler · 2022-06-02T16:35:03Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

DrewScoggins · 2022-06-02T16:35:42Z

This is the commits over the range 262347c...64e05d8

DrewScoggins · 2022-06-02T16:53:01Z

Related to this regression:
dotnet/perf-autofiling-issues#5783
dotnet/perf-autofiling-issues#5782
dotnet/perf-autofiling-issues#5781 - Only ones that are IndexOf Related

ghost · 2022-06-03T15:36:20Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	170.35 μs	202.84 μs	1.19	0.01	False
Count - Duration of single invocation	14.00 ms	22.13 ms	1.58	0.07	False
Count - Duration of single invocation	11.47 ms	18.84 ms	1.64	0.05	False
Count - Duration of single invocation	359.33 μs	401.71 μs	1.12	0.01	False
Count - Duration of single invocation	136.13 μs	156.40 μs	1.15	0.01	False
Count - Duration of single invocation	13.95 ms	22.01 ms	1.58	0.08	False
Count - Duration of single invocation	115.78 μs	148.59 μs	1.28	0.01	False
Count - Duration of single invocation	214.35 μs	225.78 μs	1.05	0.00	False
Count - Duration of single invocation	85.88 μs	117.71 μs	1.37	0.01	False
Count - Duration of single invocation	90.20 μs	123.32 μs	1.37	0.01	False
Count - Duration of single invocation	141.57 μs	177.60 μs	1.25	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 202.84127705627705 > 178.70211256270906.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 3/17/2022 6:54:53 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -177.8589860518688 (T) = (0 -202849.65972862608) / Math.Sqrt((155692.5460968798 / (33)) + (149428.62237488898 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.19489056074647046 = (169764.21639978653 - 202849.65972862608) / 169764.21639978653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.12947 > 14.690156749999998.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/19/2022 10:22:09 AM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -212.91215108523403 (T) = (0 -22190245.87708588) / Math.Sqrt((12541864947.533058 / (20)) + (16473521910.62474 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.604644335031837 = (13828762.79349817 - 22190245.87708588) / 13828762.79349817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.840791666666664 > 12.045469038461539.
IsChangePoint: Marked as a change because one of 4/19/2022 10:22:09 AM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -132.64493146334232 (T) = (0 -18908716.665649164) / Math.Sqrt((13902977425.303988 / (20)) + (44054699498.343704 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.6481416154247686 = (11472749.96801528 - 18908716.665649164) / 11472749.96801528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 401.70645264847514 > 376.95830819037076.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -20.164200721246957 (T) = (0 -393637.95899411716) / Math.Sqrt((14259805.299575618 / (20)) + (44260431.42444706 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.10039004541073393 = (357725.84515446704 - 393637.95899411716) / 357725.84515446704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.39642633228843 > 142.97913152173916.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/19/2022 10:22:09 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -23.16142001782066 (T) = (0 -157302.23190307943) / Math.Sqrt((33030007.745005228 / (33)) + (1462936.1060213188 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.20114451640660536 = (130960.28808729147 - 157302.23190307943) / 130960.28808729147 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.00644666666667 > 14.614975180555557.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 4/19/2022 10:22:09 AM, 5/27/2022 6:33:44 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -239.19249976789942 (T) = (0 -22111815.36782662) / Math.Sqrt((9111932007.963709 / (20)) + (13517419130.350264 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.6019209966703583 = (13803312.03210814 - 22111815.36782662) / 13803312.03210814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.59220238095236 > 121.26854693244485.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -152.93599467207557 (T) = (0 -148167.83849206349) / Math.Sqrt((91989.7227325268 / (33)) + (215972.8188658445 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.2842495087101177 = (115373.09338033634 - 148167.83849206349) / 115373.09338033634 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 225.7829285714286 > 225.3496136558219.
IsChangePoint: Marked as a change because one of 2/15/2022 12:48:49 AM, 2/28/2022 3:30:29 AM, 4/16/2022 8:46:33 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -84.39154468580743 (T) = (0 -225545.31332815735) / Math.Sqrt((320327.3906533836 / (33)) + (33569.01611444854 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.05036162409451314 = (214731.10608224434 - 225545.31332815735) / 214731.10608224434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.70799185463659 > 90.13683205212648.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -476.7137241354218 (T) = (0 -117618.80106860558) / Math.Sqrt((54350.887999398336 / (33)) + (14598.745742718116 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3772115474517505 = (85403.58326666315 - 117618.80106860558) / 85403.58326666315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.31891404199474 > 94.92197886560695.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -251.56982118225088 (T) = (0 -123098.91966629171) / Math.Sqrt((88242.72862351728 / (33)) + (73013.34592780822 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3672739493667976 = (90032.3740705368 - 123098.91966629171) / 90032.3740705368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 177.59994297933002 > 148.42813572464257.
IsChangePoint: Marked as a change because one of 2/12/2022 1:03:18 PM, 2/18/2022 6:05:18 PM, 4/19/2022 10:22:09 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -88.84000320256142 (T) = (0 -177886.44049185622) / Math.Sqrt((4101888.2189159435 / (33)) + (156271.74844279446 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.24528786142125564 = (142847.64671907522 - 177886.44049185622) / 142847.64671907522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfAnyThreeValues - Duration of single invocation	25.12 ns	36.48 ns	1.45	0.21	False
LastIndexOfAnyValues - Duration of single invocation	25.55 ns	35.12 ns	1.37	0.22	False
IndexOfAnyTwoValues - Duration of single invocation	20.93 ns	34.46 ns	1.65	0.20	False
LastIndexOfValue - Duration of single invocation	18.41 ns	25.38 ns	1.38	0.24	False
IndexOfValue - Duration of single invocation	16.19 ns	23.48 ns	1.45	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.48333542747834 > 26.578851645102183.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -95.6372254266481 (T) = (0 -36.39163999209435) / Math.Sqrt((0.360819912230195 / (33)) + (0.014631029646441149 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.44799960970261943 = (25.132354835073624 - 36.39163999209435) / 25.132354835073624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.117858079786885 > 26.92373391220861.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -56.523029703634535 (T) = (0 -35.26292345204584) / Math.Sqrt((0.7999462918984083 / (33)) + (0.02373966184242386 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3753482378502403 = (25.63926900954489 - 35.26292345204584) / 25.63926900954489 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.46131285641188 > 21.89515796744658.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -133.9092095156149 (T) = (0 -34.3675363500865) / Math.Sqrt((0.07690242307970953 / (33)) + (0.03900784982898168 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.6452802487516671 = (20.888560703358824 - 34.3675363500865) / 20.888560703358824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.38491067501782 > 19.32052285948416.
IsChangePoint: Marked as a change because one of 2/23/2022 2:01:04 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -90.23805183039804 (T) = (0 -25.336819571786826) / Math.Sqrt((0.14746862069050773 / (33)) + (0.007930948957717818 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.38338540445275826 = (18.31508377219695 - 25.336819571786826) / 18.31508377219695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.47667844509549 > 16.999061106314183.
IsChangePoint: Marked as a change because one of 4/8/2022 2:40:20 PM, 4/14/2022 10:59:17 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -92.95898376011084 (T) = (0 -23.37895094740987) / Math.Sqrt((0.12489434891207174 / (33)) + (0.011613701574718261 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.45082830320150435 = (16.114209307758994 - 23.37895094740987) / 16.114209307758994 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Utf8JsonReaderCommentParsing - Duration of single invocation	284.28 ns	367.26 ns	1.29	0.00	False
Utf8JsonReaderCommentParsing - Duration of single invocation	778.17 ns	853.28 ns	1.10	0.09	False
Utf8JsonReaderCommentParsing - Duration of single invocation	282.10 ns	367.97 ns	1.30	0.00	False
Utf8JsonReaderCommentParsing - Duration of single invocation	818.34 ns	899.92 ns	1.10	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 367.2607359541503 > 295.7002994826678.
IsChangePoint: Marked as a change because one of 4/12/2022 9:30:02 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -89.11344484490607 (T) = (0 -368.1217465132745) / Math.Sqrt((8.516265264479829 / (33)) + (3.6007944707960386 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.31479682896798505 = (279.9837498864535 - 368.1217465132745) / 279.9837498864535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 853.2784780578896 > 816.3021910778831.
IsChangePoint: Marked as a change because one of 4/12/2022 9:30:02 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -33.80329864409164 (T) = (0 -857.5544626935844) / Math.Sqrt((97.21382851091069 / (33)) + (29.291008658107828 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.13245218249484803 = (757.2544571412716 - 857.5544626935844) / 757.2544571412716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 367.96651018691074 > 294.9228501471663.
IsChangePoint: Marked as a change because one of 4/12/2022 9:30:02 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -172.2474661442676 (T) = (0 -366.2352230020873) / Math.Sqrt((1.9874780311813227 / (33)) + (0.9557291908109937 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.30856518870985544 = (279.8754132861864 - 366.2352230020873) / 279.8754132861864 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 899.9235548096036 > 857.5916561268206.
IsChangePoint: Marked as a change because one of 3/21/2022 10:36:51 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -26.160620230156795 (T) = (0 -896.8045289205475) / Math.Sqrt((107.50742051535703 / (33)) + (55.74403807983959 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.12450684236068399 = (797.5091792575315 - 896.8045289205475) / 797.5091792575315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	c9742b32bae35fd546f6ed844d48ccf195626ba7
Compare	06da0fcf4573115aac76f4afd65cdd75592e86e2
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	926.23 μs	1.22 ms	1.32	0.01	False
Count - Duration of single invocation	849.58 μs	1.15 ms	1.35	0.01	True
Count - Duration of single invocation	929.96 μs	1.23 ms	1.32	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w.+-]+@[\w.-]+.[\w.-]+", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2235153846153846 > 979.5645753676471.
IsChangePoint: Marked as a change because one of 3/16/2022 7:30:14 AM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -142.04426056061916 (T) = (0 -1222350.4863200902) / Math.Sqrt((66460442.677282654 / (33)) + (10514080.77879062 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.30852300283045225 = (934145.2031611495 - 1222350.4863200902) / 934145.2031611495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1485574617346936 > 890.3775032894736.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -157.51298595488143 (T) = (0 -1147964.944302721) / Math.Sqrt((98927024.773689 / (33)) + (1595280.1406405908 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.3331392660864016 = (861099.0415672901 - 1147964.944302721) / 861099.0415672901 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2314026275704488 > 983.2405719235186.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 5/31/2022 5:02:26 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsRegressionStdDev: Marked as regression because -80.77230352147373 (T) = (0 -1232654.619637331) / Math.Sqrt((238218922.30535525 / (33)) + (27360401.702412676 / (5))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (5) - 2, .025) and -0.30452099705913704 = (944909.7580002017 - 1232654.619637331) / 944909.7580002017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	tannergooding
Labels:	`tenet-performance-benchmarks`, `area-CodeGen-coreclr`, `untriaged`, `tenet-build-performance`, `refs/heads/main`, `RunKind=micro`, `Windows 10.0.19041`, `Regression`, `CoreClr`, `arm64`
Milestone:	-

tannergooding · 2022-06-16T19:09:23Z

Most of these were resolved in 2d5c8dc

A few were mostly fixed but still have a small regression related to #70182

performanceautofiler bot assigned tannergooding Jun 2, 2022

performanceautofiler bot added arm64 untriaged New issue has not been triaged by the area owner labels Jun 2, 2022

ghost added the untriaged New issue has not been triaged by the area owner label Jun 2, 2022

DrewScoggins transferred this issue from dotnet/perf-autofiling-issues Jun 2, 2022

DrewScoggins added tenet-performance-benchmarks Issue from performance benchmark tenet-build-performance Impacts build time: official, developer or CI labels Jun 2, 2022

DrewScoggins changed the title ~~[Perf] Changes at 5/31/2022 10:14:36 PM~~ [Perf] Regressions in IndexOf Jun 2, 2022

jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jun 3, 2022

JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label Jun 3, 2022

JulieLeeMSFT added this to the 7.0.0 milestone Jun 3, 2022

tannergooding closed this as completed Jun 16, 2022

ghost locked as resolved and limited conversation to collaborators Jul 17, 2022

jeffhandley added arch-arm64 runtime-coreclr specific to the CoreCLR runtime and removed arm64 labels Dec 28, 2022

jeffhandley removed the Regression label Dec 28, 2022

[Perf] Regressions in IndexOf #70158

[Perf] Regressions in IndexOf #70158

Comments

performanceautofiler bot commented Jun 2, 2022

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Memory.Span<Byte>

Repro

Payloads

Histogram

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Repro

Payloads

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w.+-]+@[\w.-]+.[\w.-]+", Options: None)

Description of detection logic

Description of detection logic

Description of detection logic

Docs

performanceautofiler bot commented Jun 2, 2022

Run Information

Regressions in System.Memory.Span<Char>

Repro

Payloads

Histogram

System.Memory.Span<Char>.IndexOfValue(Size: 512)

Description of detection logic

Description of detection logic

Docs

Regressions in System.Tests.Perf_Array

Repro

Payloads

Histogram

System.Tests.Perf_Array.IndexOfChar

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Memory.ReadOnlySpan

Repro

Payloads

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

Description of detection logic

DrewScoggins commented Jun 2, 2022 •

edited

Loading