[Perf] Changes at 5/28/2022 1:41:21 AM #70022

performanceautofiler · 2022-05-31T12:22:40Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2a6613530b19fafb34c2c1dd731f7e4672d3f53c
Compare	e297470559fb95344ab52c8dad561885a1430e7e
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IsMatch - Duration of single invocation	169.40 ns	220.44 ns	1.30	0.02	True
IsMatch - Duration of single invocation	212.68 ns	266.10 ns	1.25	0.05	True
IsMatch - Duration of single invocation	94.92 ns	110.11 ns	1.16	0.02	True
IsMatch - Duration of single invocation	173.82 ns	192.99 ns	1.11	0.02	False
IsMatch - Duration of single invocation	157.16 ns	210.37 ns	1.34	0.01	True
IsMatch - Duration of single invocation	117.57 ns	129.88 ns	1.10	0.03	False
IsMatch - Duration of single invocation	116.91 ns	130.32 ns	1.11	0.01	False
IsMatch - Duration of single invocation	148.51 ns	168.20 ns	1.13	0.03	True
IsMatch - Duration of single invocation	88.98 ns	123.63 ns	1.39	0.03	True
IsMatch - Duration of single invocation	75.08 ns	95.67 ns	1.27	0.02	True
IsMatch - Duration of single invocation	146.92 ns	168.49 ns	1.15	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 220.44179117581695 > 176.759814121078.
IsChangePoint: Marked as a change because one of 4/19/2022 10:22:09 AM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -53.70088388499484 (T) = (0 -224.47618023633657) / Math.Sqrt((3.0371132269902166 / (50)) + (16.216177842847408 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.3296992492747151 = (168.81725725480945 - 224.47618023633657) / 168.81725725480945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 266.0985181753315 > 223.49439389954148.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -32.365883774389495 (T) = (0 -271.63461544263737) / Math.Sqrt((9.938813341166195 / (50)) + (46.51474138098469 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.2658073865514193 = (214.59395665456017 - 271.63461544263737) / 214.59395665456017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.10618462752404 > 100.53190206655806.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -44.7354529268721 (T) = (0 -111.77654956412448) / Math.Sqrt((2.3924697878150374 / (50)) + (1.086760875208175 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.15764416547439075 = (96.55518759369343 - 111.77654956412448) / 96.55518759369343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 192.99371784109994 > 184.2750428404849.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -33.164131929245606 (T) = (0 -194.2872620956936) / Math.Sqrt((5.229029925663083 / (50)) + (2.6929088304216 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.09789904499940576 = (176.9627753850526 - 194.2872620956936) / 176.9627753850526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.36558264247532 > 165.89695201471514.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -46.14007808353078 (T) = (0 -214.64030501804936) / Math.Sqrt((3.0595256874581493 / (50)) + (22.7075780958421 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.35419217385154045 = (158.50062432983776 - 214.64030501804936) / 158.50062432983776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.8780734443098 > 123.28032488088854.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -28.085464931996988 (T) = (0 -131.2426011556937) / Math.Sqrt((4.750785374586515 / (49)) + (1.4698886227499244 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.102523105718044 = (119.0384133221579 - 131.2426011556937) / 119.0384133221579 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.3224630464478 > 123.1618783745977.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -16.420213513981253 (T) = (0 -130.68447831303436) / Math.Sqrt((20.900065439245687 / (50)) + (0.7486242502689555 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.09368630362302288 = (119.48991029705655 - 130.68447831303436) / 119.48991029705655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.20095911260492 > 155.28948610908444.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -9.68887840681239 (T) = (0 -169.8484994864781) / Math.Sqrt((2.009576966132443 / (50)) + (78.43329159627501 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.14523440544206914 = (148.30893892060053 - 169.8484994864781) / 148.30893892060053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.63044395398607 > 93.0842515744019.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -96.84281235936886 (T) = (0 -124.93193935171097) / Math.Sqrt((1.619419789626349 / (49)) + (1.6665360443327422 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.4027866559914532 = (89.05982874738163 - 124.93193935171097) / 89.05982874738163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.66911985412614 > 79.87331574778298.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -43.72119918193564 (T) = (0 -95.2017052804285) / Math.Sqrt((1.3977098462922453 / (49)) + (2.579590949941682 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.25007717462623696 = (76.15666233478188 - 95.2017052804285) / 76.15666233478188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.48929606235987 > 156.37934021166922.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -51.83332556725939 (T) = (0 -167.22629232595915) / Math.Sqrt((2.1702643578951 / (50)) + (1.449933208972095 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.12799945260253823 = (148.25033109735293 - 167.22629232595915) / 148.25033109735293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2a6613530b19fafb34c2c1dd731f7e4672d3f53c
Compare	e297470559fb95344ab52c8dad561885a1430e7e
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	3.63 ms	8.69 ms	2.40	0.03	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.689346478160921 > 3.807549869315611.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -115.27784204861011 (T) = (0 -8546265.076734848) / Math.Sqrt((1359463523.4742622 / (49)) + (28284723337.588894 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -1.3340857932411931 = (3661504.260675528 - 8546265.076734848) / 3661504.260675528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2a6613530b19fafb34c2c1dd731f7e4672d3f53c
Compare	e297470559fb95344ab52c8dad561885a1430e7e
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	14.83 ms	20.64 ms	1.39	0.00	True
Count - Duration of single invocation	28.47 ms	34.99 ms	1.23	0.05	False
Count - Duration of single invocation	99.99 μs	118.27 μs	1.18	0.02	False
Count - Duration of single invocation	1.28 ms	1.47 ms	1.14	0.01	False
Count - Duration of single invocation	8.18 ms	15.18 ms	1.86	0.16	False
Count - Duration of single invocation	1.32 ms	1.69 ms	1.28	0.00	True
Count - Duration of single invocation	171.18 μs	200.60 μs	1.17	0.02	False
Count - Duration of single invocation	28.41 ms	35.60 ms	1.25	0.05	False
Count - Duration of single invocation	6.71 ms	13.11 ms	1.95	0.03	False
Count - Duration of single invocation	8.21 ms	14.84 ms	1.81	0.11	False
Count - Duration of single invocation	678.81 μs	813.02 μs	1.20	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.640817983333335 > 15.789817881250002.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -141.48496677584646 (T) = (0 -20502078.218108978) / Math.Sqrt((32356388201.43153 / (49)) + (14485228674.222132 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.3756331887725005 = (14903739.154769383 - 20502078.218108978) / 14903739.154769383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.98910971666667 > 29.4539423325.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -24.49977087636941 (T) = (0 -34819778.9799437) / Math.Sqrt((3019562637362.6167 / (50)) + (169965095621.85803 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.23077377367081325 = (28290965.996206474 - 34819778.9799437) / 28290965.996206474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.26687412728252 > 104.73706699161698.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -15.209415550855512 (T) = (0 -114850.34704021331) / Math.Sqrt((1291945.2198616548 / (49)) + (13594081.648927229 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.14148171550222208 = (100615.1438787455 - 114850.34704021331) / 100615.1438787455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4660554194880262 > 1.353980954657452.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -38.64940800247274 (T) = (0 -1449806.4179791661) / Math.Sqrt((362278639.12680036 / (49)) + (79002598.22249025 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.10328087233400768 = (1314086.4256189617 - 1449806.4179791661) / 1314086.4256189617 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.177288321428572 > 8.606134079567306.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -51.4504627648497 (T) = (0 -15160024.865670789) / Math.Sqrt((657213206387.2999 / (50)) + (37185587122.02227 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.7303921301182272 = (8761034.335399454 - 15160024.865670789) / 8761034.335399454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.692994796814672 > 1.3805915936848958.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -67.9223565025587 (T) = (0 -1682597.1783218125) / Math.Sqrt((257297557.31335682 / (50)) + (362419665.6393984 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.2703737080257259 = (1324489.9258319181 - 1682597.1783218125) / 1324489.9258319181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.5988130239152 > 181.01734747081042.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -57.218068433049865 (T) = (0 -200777.43744955427) / Math.Sqrt((3596405.0618703403 / (50)) + (2181215.23818436 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.14949319437201586 = (174666.05146735278 - 200777.43744955427) / 174666.05146735278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.600647035714275 > 29.972355125781245.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -43.98449332246277 (T) = (0 -35861946.387808666) / Math.Sqrt((368155205964.1368 / (50)) + (293646927467.4926 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.24483924367937593 = (28808496.012554504 - 35861946.387808666) / 28808496.012554504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.10944429959514 > 7.051958976621622.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -171.19850252112573 (T) = (0 -13052238.826462792) / Math.Sqrt((12572256138.03449 / (50)) + (17682859348.32323 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.9346419569047767 = (6746591.41960562 - 13052238.826462792) / 6746591.41960562 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.836153900000001 > 8.644265332500002.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -43.73122820574829 (T) = (0 -14968422.951865843) / Math.Sqrt((830190521191.0114 / (50)) + (39318967772.06097 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.6760484971373009 = (8930781.52418142 - 14968422.951865843) / 8930781.52418142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 813.0216045016077 > 713.1578000883152.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -40.248978694116126 (T) = (0 -794512.638510615) / Math.Sqrt((58387816.280361116 / (49)) + (110773638.72577962 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.16864765953399383 = (679856.440928853 - 794512.638510615) / 679856.440928853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

tannergooding · 2022-05-31T16:25:45Z

Possibly related #69773

ghost · 2022-05-31T16:26:01Z

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2a6613530b19fafb34c2c1dd731f7e4672d3f53c
Compare	e297470559fb95344ab52c8dad561885a1430e7e
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IsMatch - Duration of single invocation	169.40 ns	220.44 ns	1.30	0.02	True
IsMatch - Duration of single invocation	212.68 ns	266.10 ns	1.25	0.05	True
IsMatch - Duration of single invocation	94.92 ns	110.11 ns	1.16	0.02	True
IsMatch - Duration of single invocation	173.82 ns	192.99 ns	1.11	0.02	False
IsMatch - Duration of single invocation	157.16 ns	210.37 ns	1.34	0.01	True
IsMatch - Duration of single invocation	117.57 ns	129.88 ns	1.10	0.03	False
IsMatch - Duration of single invocation	116.91 ns	130.32 ns	1.11	0.01	False
IsMatch - Duration of single invocation	148.51 ns	168.20 ns	1.13	0.03	True
IsMatch - Duration of single invocation	88.98 ns	123.63 ns	1.39	0.03	True
IsMatch - Duration of single invocation	75.08 ns	95.67 ns	1.27	0.02	True
IsMatch - Duration of single invocation	146.92 ns	168.49 ns	1.15	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 220.44179117581695 > 176.759814121078.
IsChangePoint: Marked as a change because one of 4/19/2022 10:22:09 AM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -53.70088388499484 (T) = (0 -224.47618023633657) / Math.Sqrt((3.0371132269902166 / (50)) + (16.216177842847408 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.3296992492747151 = (168.81725725480945 - 224.47618023633657) / 168.81725725480945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 266.0985181753315 > 223.49439389954148.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -32.365883774389495 (T) = (0 -271.63461544263737) / Math.Sqrt((9.938813341166195 / (50)) + (46.51474138098469 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.2658073865514193 = (214.59395665456017 - 271.63461544263737) / 214.59395665456017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.10618462752404 > 100.53190206655806.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -44.7354529268721 (T) = (0 -111.77654956412448) / Math.Sqrt((2.3924697878150374 / (50)) + (1.086760875208175 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.15764416547439075 = (96.55518759369343 - 111.77654956412448) / 96.55518759369343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 192.99371784109994 > 184.2750428404849.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -33.164131929245606 (T) = (0 -194.2872620956936) / Math.Sqrt((5.229029925663083 / (50)) + (2.6929088304216 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.09789904499940576 = (176.9627753850526 - 194.2872620956936) / 176.9627753850526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.36558264247532 > 165.89695201471514.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -46.14007808353078 (T) = (0 -214.64030501804936) / Math.Sqrt((3.0595256874581493 / (50)) + (22.7075780958421 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.35419217385154045 = (158.50062432983776 - 214.64030501804936) / 158.50062432983776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.8780734443098 > 123.28032488088854.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -28.085464931996988 (T) = (0 -131.2426011556937) / Math.Sqrt((4.750785374586515 / (49)) + (1.4698886227499244 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.102523105718044 = (119.0384133221579 - 131.2426011556937) / 119.0384133221579 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.3224630464478 > 123.1618783745977.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -16.420213513981253 (T) = (0 -130.68447831303436) / Math.Sqrt((20.900065439245687 / (50)) + (0.7486242502689555 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.09368630362302288 = (119.48991029705655 - 130.68447831303436) / 119.48991029705655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.20095911260492 > 155.28948610908444.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -9.68887840681239 (T) = (0 -169.8484994864781) / Math.Sqrt((2.009576966132443 / (50)) + (78.43329159627501 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.14523440544206914 = (148.30893892060053 - 169.8484994864781) / 148.30893892060053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.63044395398607 > 93.0842515744019.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -96.84281235936886 (T) = (0 -124.93193935171097) / Math.Sqrt((1.619419789626349 / (49)) + (1.6665360443327422 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.4027866559914532 = (89.05982874738163 - 124.93193935171097) / 89.05982874738163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.66911985412614 > 79.87331574778298.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -43.72119918193564 (T) = (0 -95.2017052804285) / Math.Sqrt((1.3977098462922453 / (49)) + (2.579590949941682 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.25007717462623696 = (76.15666233478188 - 95.2017052804285) / 76.15666233478188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.48929606235987 > 156.37934021166922.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -51.83332556725939 (T) = (0 -167.22629232595915) / Math.Sqrt((2.1702643578951 / (50)) + (1.449933208972095 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.12799945260253823 = (148.25033109735293 - 167.22629232595915) / 148.25033109735293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2a6613530b19fafb34c2c1dd731f7e4672d3f53c
Compare	e297470559fb95344ab52c8dad561885a1430e7e
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	3.63 ms	8.69 ms	2.40	0.03	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.689346478160921 > 3.807549869315611.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -115.27784204861011 (T) = (0 -8546265.076734848) / Math.Sqrt((1359463523.4742622 / (49)) + (28284723337.588894 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -1.3340857932411931 = (3661504.260675528 - 8546265.076734848) / 3661504.260675528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	2a6613530b19fafb34c2c1dd731f7e4672d3f53c
Compare	e297470559fb95344ab52c8dad561885a1430e7e
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	14.83 ms	20.64 ms	1.39	0.00	True
Count - Duration of single invocation	28.47 ms	34.99 ms	1.23	0.05	False
Count - Duration of single invocation	99.99 μs	118.27 μs	1.18	0.02	False
Count - Duration of single invocation	1.28 ms	1.47 ms	1.14	0.01	False
Count - Duration of single invocation	8.18 ms	15.18 ms	1.86	0.16	False
Count - Duration of single invocation	1.32 ms	1.69 ms	1.28	0.00	True
Count - Duration of single invocation	171.18 μs	200.60 μs	1.17	0.02	False
Count - Duration of single invocation	28.41 ms	35.60 ms	1.25	0.05	False
Count - Duration of single invocation	6.71 ms	13.11 ms	1.95	0.03	False
Count - Duration of single invocation	8.21 ms	14.84 ms	1.81	0.11	False
Count - Duration of single invocation	678.81 μs	813.02 μs	1.20	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.640817983333335 > 15.789817881250002.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -141.48496677584646 (T) = (0 -20502078.218108978) / Math.Sqrt((32356388201.43153 / (49)) + (14485228674.222132 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.3756331887725005 = (14903739.154769383 - 20502078.218108978) / 14903739.154769383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.98910971666667 > 29.4539423325.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -24.49977087636941 (T) = (0 -34819778.9799437) / Math.Sqrt((3019562637362.6167 / (50)) + (169965095621.85803 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.23077377367081325 = (28290965.996206474 - 34819778.9799437) / 28290965.996206474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.26687412728252 > 104.73706699161698.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -15.209415550855512 (T) = (0 -114850.34704021331) / Math.Sqrt((1291945.2198616548 / (49)) + (13594081.648927229 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.14148171550222208 = (100615.1438787455 - 114850.34704021331) / 100615.1438787455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4660554194880262 > 1.353980954657452.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -38.64940800247274 (T) = (0 -1449806.4179791661) / Math.Sqrt((362278639.12680036 / (49)) + (79002598.22249025 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.10328087233400768 = (1314086.4256189617 - 1449806.4179791661) / 1314086.4256189617 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.177288321428572 > 8.606134079567306.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -51.4504627648497 (T) = (0 -15160024.865670789) / Math.Sqrt((657213206387.2999 / (50)) + (37185587122.02227 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.7303921301182272 = (8761034.335399454 - 15160024.865670789) / 8761034.335399454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.692994796814672 > 1.3805915936848958.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -67.9223565025587 (T) = (0 -1682597.1783218125) / Math.Sqrt((257297557.31335682 / (50)) + (362419665.6393984 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.2703737080257259 = (1324489.9258319181 - 1682597.1783218125) / 1324489.9258319181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.5988130239152 > 181.01734747081042.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -57.218068433049865 (T) = (0 -200777.43744955427) / Math.Sqrt((3596405.0618703403 / (50)) + (2181215.23818436 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.14949319437201586 = (174666.05146735278 - 200777.43744955427) / 174666.05146735278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.600647035714275 > 29.972355125781245.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -43.98449332246277 (T) = (0 -35861946.387808666) / Math.Sqrt((368155205964.1368 / (50)) + (293646927467.4926 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.24483924367937593 = (28808496.012554504 - 35861946.387808666) / 28808496.012554504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.10944429959514 > 7.051958976621622.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -171.19850252112573 (T) = (0 -13052238.826462792) / Math.Sqrt((12572256138.03449 / (50)) + (17682859348.32323 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.9346419569047767 = (6746591.41960562 - 13052238.826462792) / 6746591.41960562 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.836153900000001 > 8.644265332500002.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -43.73122820574829 (T) = (0 -14968422.951865843) / Math.Sqrt((830190521191.0114 / (50)) + (39318967772.06097 / (16))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (16) - 2, .025) and -0.6760484971373009 = (8930781.52418142 - 14968422.951865843) / 8930781.52418142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 813.0216045016077 > 713.1578000883152.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/27/2022 11:47:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -40.248978694116126 (T) = (0 -794512.638510615) / Math.Sqrt((58387816.280361116 / (49)) + (110773638.72577962 / (16))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (16) - 2, .025) and -0.16864765953399383 = (679856.440928853 - 794512.638510615) / 679856.440928853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	tannergooding
Labels:	`area-System.Text.RegularExpressions`, `untriaged`, `refs/heads/main`, `ubuntu 18.04`, `RunKind=micro`, `Regression`, `CoreClr`, `x64`
Milestone:	-

stephentoub · 2022-05-31T16:33:39Z

Possibly related #69773

All of these regressions are with RegexOptions.NonBacktracking, which isn't affected by the source generator.

If the diff list of commits is accurate, the most likely candidate is #69839.

@olsaarik, something regressed NonBacktracking on a bunch of benchmarks fairly significantly. Did you expect your #69839 to have any impact on matching throughput?

olsaarik · 2022-05-31T20:06:12Z

@stephentoub No I did not expect #69839 to have any negative impact on matching throughput. I will investigate.

olsaarik · 2022-06-02T02:11:22Z

Just an update, one culprit at least was an additional call to GetCharKind inside FindEndPositionDeltas in a code path where it finds a candidate end state, which is a hot code path for some patterns. I'm doing a round of additional investigations to improve the performance of the inner matching loops. There's other checks like IsDeadend and IsInitialState in there that might be avoidable in some situations.

olsaarik · 2022-06-07T18:43:43Z

This is hopefully mostly fixed in #70217. Would anyone know how I could look at what those graphs look like now to verify that the regression is gone?

stephentoub · 2022-06-09T13:57:33Z

Would anyone know how I could look at what those graphs look like now to verify that the regression is gone?

Click on the link to the test in the original post above:

and then on the resulting page, click on the graph on the right-hand side middle:

That'll bring up a larger graph showing up-to-date history, including at least one run with your changes included:

stephentoub · 2022-06-09T13:58:54Z

This pages also shows the history for all of our regex tests:
https://pvscmdupload.blob.core.windows.net/reports/06_08_2022/Regex/report_Daily_ca=x64_cb=refs-heads-main_co=Windows1018362_cr=dotnetruntime_cf=Regex_cc=CompliationMode=tiered-RunKind=micro_2022-06-08.html

stephentoub · 2022-07-12T14:33:08Z

Most of these have recovered or improved (or recovered to acceptable levels).

performanceautofiler bot assigned tannergooding May 31, 2022

performanceautofiler bot added CoreClr untriaged New issue has not been triaged by the area owner labels May 31, 2022

dotnet-issue-labeler bot added the area-System.Text.RegularExpressions label May 31, 2022

tannergooding transferred this issue from dotnet/perf-autofiling-issues May 31, 2022

olsaarik self-assigned this May 31, 2022

olsaarik mentioned this issue Jun 3, 2022

NonBacktracking inner matching loop optimizations #70217

Merged

joperezr added this to the 7.0.0 milestone Jun 15, 2022

ghost removed the untriaged New issue has not been triaged by the area owner label Jun 15, 2022

stephentoub closed this as completed Jul 12, 2022

ghost locked as resolved and limited conversation to collaborators Aug 11, 2022

jeffhandley added runtime-coreclr specific to the CoreCLR runtime and removed CoreClr labels Dec 28, 2022

jeffhandley added os-linux Linux OS (any supported distro) arch-x64 and removed ubuntu 18.04 labels Dec 28, 2022

[Perf] Changes at 5/28/2022 1:41:21 AM #70022

[Perf] Changes at 5/28/2022 1:41:21 AM #70022

Comments

performanceautofiler bot commented May 31, 2022

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)

Description of detection logic

Docs

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

tannergooding commented May 31, 2022

ghost commented May 31, 2022

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:?[^\s#])?(?:#[^\s])?", Options: NonBacktracking)

Description of detection logic

Docs

Run Information

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Repro

Payloads

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking)

stephentoub commented Jun 9, 2022 •

edited

Loading

stephentoub commented Jun 9, 2022 •

edited

Loading