Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 16 Regressions on 9/22/2022 3:16:12 AM #77900

Closed
performanceautofiler bot opened this issue Sep 27, 2022 · 29 comments · Fixed by dotnet/performance#2703
Closed

[Perf] Linux/x64: 16 Regressions on 9/22/2022 3:16:12 AM #77900

performanceautofiler bot opened this issue Sep 27, 2022 · 29 comments · Fixed by dotnet/performance#2703
Assignees
@jeffhandley @adamsitnik @kotlarmilos
Labels
arch-x64 area-System.Memory os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 969.31 μs 1.10 ms 1.13 0.07 False
ImmutableArray - Duration of single invocation 996.73 μs 1.11 ms 1.12 0.05 False
ICollection - Duration of single invocation 967.59 μs 1.06 ms 1.10 0.05 False
List - Duration of single invocation 954.73 μs 1.06 ms 1.11 0.06 False
Array - Duration of single invocation 974.99 μs 1.07 ms 1.10 0.06 False
Stack - Duration of single invocation 1.00 ms 1.24 ms 1.23 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.095966200892857 > 1010.798569984976.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -26.205242060142314 (T) = (0 -1082176.5961467223) / Math.Sqrt((134557800.542629 / (33)) + (378753722.89546967 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11677096832909976 = (969022.8586134029 - 1082176.5961467223) / 969022.8586134029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1139794651785715 > 1.060484347207031.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -27.791974606900073 (T) = (0 -1115382.3246755877) / Math.Sqrt((180256514.8127183 / (33)) + (235012253.979763 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10483220306009597 = (1009549.0714212264 - 1115382.3246755877) / 1009549.0714212264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0617160416666667 > 1018.8446187011718.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -16.056018293701424 (T) = (0 -1087710.4426824627) / Math.Sqrt((185033592.69745567 / (33)) + (1162193806.342326 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11694145311763579 = (973829.4157195234 - 1087710.4426824627) / 973829.4157195234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0611418856944443 > 1018.2846668750002.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -35.482093782068404 (T) = (0 -1074002.2283184354) / Math.Sqrt((69967824.32398583 / (33)) + (179800464.4794185 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11025724102303357 = (967345.3940537317 - 1074002.2283184354) / 967345.3940537317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0702191419642857 > 1023.6990125195313.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -30.120564854575687 (T) = (0 -1086000.919079096) / Math.Sqrt((291634961.6495431 / (33)) + (86090478.21488371 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10701630982952783 = (981016.1868765348 - 1086000.919079096) / 981016.1868765348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2352883976648354 > 1024.4168104882813.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -60.858087292552526 (T) = (0 -1236941.1502558445) / Math.Sqrt((337610251.7351471 / (33)) + (203005793.13338426 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2641636960719056 = (978465.9645735368 - 1236941.1502558445) / 978465.9645735368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 4.81 μs 5.38 μs 1.12 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.378923124141483 > 5.053520181621476.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -51.38447919574725 (T) = (0 -5367.488921550735) / Math.Sqrt((2937.7007674825045 / (32)) + (472.35950356068855 / (26))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (26) - 2, .025) and -0.11159539603981007 = (4828.635437563927 - 5367.488921550735) / 4828.635437563927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 1.89 ms 2.38 ms 1.26 0.06 False
ImmutableArray - Duration of single invocation 1.93 ms 2.14 ms 1.11 0.03 False
Array - Duration of single invocation 1.90 ms 2.11 ms 1.11 0.03 False
List - Duration of single invocation 1.89 ms 2.10 ms 1.11 0.05 False
Queue - Duration of single invocation 1.89 ms 2.23 ms 1.18 0.06 False
ICollection - Duration of single invocation 1.87 ms 2.10 ms 1.12 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3842154708333334 > 1.9866159997796478.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -63.97195320199993 (T) = (0 -2394094.2410446103) / Math.Sqrt((1695724547.8362103 / (33)) + (190835740.57041073 / (25))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (25) - 2, .025) and -0.2583031159565782 = (1902637.1394022887 - 2394094.2410446103) / 1902637.1394022887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1394566171875 > 2.0256352184960935.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -31.077029944721016 (T) = (0 -2151332.1590478206) / Math.Sqrt((360162614.46956015 / (33)) + (997867569.4428535 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11286796194600314 = (1933142.3247066252 - 2151332.1590478206) / 1933142.3247066252 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1087881843750003 > 1.9809466521310766.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -45.259498036275815 (T) = (0 -2113783.74757238) / Math.Sqrt((353745729.22009766 / (33)) + (319361059.55712956 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11444507736727372 = (1896714.1499390076 - 2113783.74757238) / 1896714.1499390076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1026052706473215 > 1.9608499316145835.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -68.80938666641624 (T) = (0 -2099614.6035013716) / Math.Sqrt((211620946.8627083 / (33)) + (92498111.32462098 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11542629051873408 = (1882342.7611025164 - 2099614.6035013716) / 1882342.7611025164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2304079871651785 > 2.008788108359375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -23.59913129326589 (T) = (0 -2147578.562214543) / Math.Sqrt((1116474284.4023762 / (33)) + (1587306697.1320746 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11986933064211758 = (1917704.5959309835 - 2147578.562214543) / 1917704.5959309835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0979382890625 > 1.9806727393359376.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -42.85148744252998 (T) = (0 -2103699.1899045617) / Math.Sqrt((347858606.2090312 / (33)) + (396049417.258107 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11533974964320046 = (1886151.005178054 - 2103699.1899045617) / 1886151.005178054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 832.17 ns 895.38 ns 1.08 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 895.3778104156972 > 849.9629668568364.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -29.21081282954959 (T) = (0 -892.9335262280066) / Math.Sqrt((168.6694102042165 / (33)) + (82.54061160444941 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10395462812604744 = (808.8498417219853 - 892.9335262280066) / 808.8498417219853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 1.87 μs 2.49 μs 1.33 0.06 False
IndexOfValue - Duration of single invocation 1.98 μs 2.11 μs 1.07 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.486075723145284 > 1.961551330630418.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -32.74534469892047 (T) = (0 -2378.1927078714825) / Math.Sqrt((666.1354551089895 / (33)) + (5810.079235983002 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2737630503998014 = (1867.0605236390152 - 2378.1927078714825) / 1867.0605236390152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1059525350629564 > 1.9711226404728943.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -26.75366215584882 (T) = (0 -2108.271736496125) / Math.Sqrt((824.5871390603775 / (33)) + (1462.8924148464291 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.1291609019154783 = (1867.1136530849672 - 2108.271736496125) / 1867.1136530849672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added refs/heads/main untriaged New issue has not been triaged by the area owner labels Sep 27, 2022
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 47.13 ns 390.48 ns 8.29 0.05 False
IndexOfString - Duration of single invocation 360.94 ns 3.68 μs 10.19 0.04 False
IndexOfString - Duration of single invocation 28.44 ns 157.52 ns 5.54 0.09 False
IndexOfString - Duration of single invocation 47.63 ns 321.90 ns 6.76 0.03 False
IndexOfString - Duration of single invocation 358.23 ns 3.66 μs 10.22 0.04 False
IndexOfString - Duration of single invocation 130.51 ns 969.46 ns 7.43 0.12 False
IndexOfString - Duration of single invocation 149.19 ns 1.37 μs 9.18 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 390.48395692708334 > 49.51093434422919.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -142.21634422379435 (T) = (0 -387.6700858236727) / Math.Sqrt((0.9803100425690886 / (33)) + (148.03039521274317 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -7.171206230990526 = (47.443434281878204 - 387.6700858236727) / 47.443434281878204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6772498289397864 > 377.00408117106747.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -91.80204807734657 (T) = (0 -3643.760145406374) / Math.Sqrt((10.21797841315963 / (33)) + (33269.25016197939 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -9.13610052557203 = (359.4834262163889 - 3643.760145406374) / 359.4834262163889 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.51741786388365 > 29.058065086960717.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 6:35:49 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -94.55913552755942 (T) = (0 -146.12517890755333) / Math.Sqrt((0.7632621402676609 / (32)) + (40.264270168228 / (26))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (26) - 2, .025) and -4.304101980554208 = (27.54946632686825 - 146.12517890755333) / 27.54946632686825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 321.9039432976289 > 49.97405924635684.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -158.51083116205075 (T) = (0 -317.1568498077676) / Math.Sqrt((0.8999623168663393 / (33)) + (74.33386014875505 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -5.626407202744838 = (47.862565656452965 - 317.1568498077676) / 47.862565656452965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.660039173054245 > 376.9835614442801.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -151.98635143984134 (T) = (0 -3608.6277821709564) / Math.Sqrt((10.811975181020898 / (33)) + (11867.252960508345 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -9.013330091261075 = (360.38238520872403 - 3608.6277821709564) / 360.38238520872403 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 969.4579645817847 > 137.15697184671575.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -87.67488863413676 (T) = (0 -1033.509664145777) / Math.Sqrt((1.1063218804754364 / (33)) + (2754.078602939491 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -6.888581880729497 = (131.01336587130703 - 1033.509664145777) / 131.01336587130703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3688763789542615 > 154.81829836334515.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -193.30585675684685 (T) = (0 -1348.8352081259331) / Math.Sqrt((2.1698436376796746 / (33)) + (1001.4022996862954 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -8.105180333279282 = (148.13931836099533 - 1348.8352081259331) / 148.13931836099533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 236.70 ns 1.45 μs 6.13 0.12 False
IndexOf_Word_NotFound - Duration of single invocation 387.68 ns 3.05 μs 7.87 0.15 False
IndexOf_Word_NotFound - Duration of single invocation 223.76 ns 1.65 μs 7.38 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4498081786829973 > 247.53797217620658.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -112.40196317600514 (T) = (0 -1516.6395329568463) / Math.Sqrt((111.46033872490173 / (32)) + (3257.133258661372 / (26))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (26) - 2, .025) and -5.287913082088506 = (241.19918853157878 - 1516.6395329568463) / 241.19918853157878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.052055684075956 > 404.6118889928561.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -94.71929535155282 (T) = (0 -3185.5856777930135) / Math.Sqrt((29.162558114566853 / (33)) + (22717.654416434052 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -7.2887133151934895 = (384.32812870409305 - 3185.5856777930135) / 384.32812870409305 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.651578455846479 > 235.61874933610076.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -141.92407286168483 (T) = (0 -1638.292181360805) / Math.Sqrt((1.7666156449029948 / (33)) + (2577.5226811216717 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -6.287191937921552 = (224.81803626378442 - 1638.292181360805) / 224.81803626378442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfShort - Duration of single invocation 63.18 ns 499.20 ns 7.90 0.04 False
IndexOfChar - Duration of single invocation 57.65 ns 492.93 ns 8.55 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Array.IndexOfShort

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 499.20217095262217 > 66.35420770518094.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -164.2777062733905 (T) = (0 -488.60160975876903) / Math.Sqrt((0.1343330203228698 / (33)) + (174.17988721966282 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -6.721911699714787 = (63.27469527744221 - 488.60160975876903) / 63.27469527744221 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfChar

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 492.93372779428495 > 60.403250631376714.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -168.9753981037571 (T) = (0 -486.9427003570417) / Math.Sqrt((0.25235512590680653 / (33)) + (167.6276577264438 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -7.448496747083709 = (57.63660861029825 - 486.9427003570417) / 57.63660861029825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 891.33 ns 1.07 μs 1.20 0.02 False
GetName - Duration of single invocation 27.07 ns 48.58 ns 1.79 0.11 False
IsDefined - Duration of single invocation 28.78 ns 47.63 ns 1.65 0.14 False
TryParseGeneric - Duration of single invocation 766.16 ns 916.36 ns 1.20 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Enum.Parse(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0664630472585925 > 934.562105146875.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -51.071660756232866 (T) = (0 -1064.7047246098873) / Math.Sqrt((16.706572761458073 / (33)) + (287.42457506064767 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.1948834751049625 = (891.0531836724582 - 1064.7047246098873) / 891.0531836724582 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.GetName

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.577811728203905 > 27.80071223994612.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -200.97907579926627 (T) = (0 -48.59911350593771) / Math.Sqrt((0.3004787259600039 / (33)) + (0.07558975271152785 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.8290078317547709 = (26.57129874578567 - 48.59911350593771) / 26.57129874578567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.IsDefined

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.631558197597435 > 30.107237171672452.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -90.95164286729901 (T) = (0 -48.11930475505811) / Math.Sqrt((0.1755324337928859 / (33)) + (1.0921687825010407 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.6983324812906123 = (28.333265297081788 - 48.11930475505811) / 28.333265297081788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 916.3604519984391 > 804.5710769113867.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -58.17776212980565 (T) = (0 -925.3940351596938) / Math.Sqrt((27.580578792602704 / (33)) + (168.97726057310473 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.20520523695336543 = (767.8310770528964 - 925.3940351596938) / 767.8310770528964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 245.90 ns 517.55 ns 2.10 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.Formatting*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Logging.Formatting.TwoArguments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 517.5537808947925 > 257.93452465513553.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -121.20586791062122 (T) = (0 -520.8726888389632) / Math.Sqrt((11.925818205810415 / (33)) + (124.01937024707651 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.114695895409134 = (246.31091873292215 - 520.8726888389632) / 246.31091873292215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NestedScopes_TwoMessages - Duration of single invocation 1.80 μs 2.16 μs 1.20 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.16272991024049 > 1.9785593203347334.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -21.21867562311999 (T) = (0 -2217.4455827269244) / Math.Sqrt((2935.9241844756643 / (33)) + (4612.32492912376 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.18507578926747795 = (1871.1424221210168 - 2217.4455827269244) / 1871.1424221210168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 185.86 ns 873.78 ns 4.70 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Int32>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 873.7789838321005 > 198.5596598678169.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1627.0721648798205 (T) = (0 -873.3528025185454) / Math.Sqrt((3.758182013474738 / (33)) + (1.6628985709350754 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.665466992208811 = (187.19515194877988 - 873.3528025185454) / 187.19515194877988 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddChainedConfigurationEmpty - Duration of single invocation 78.79 ms 91.71 ms 1.16 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.70925511666665 > 82.829823236875.
IsChangePoint: Marked as a change because one of 9/2/2022 11:09:40 AM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -40.04903030549572 (T) = (0 -93200301.77639475) / Math.Sqrt((2745426596275.1904 / (33)) + (805561486201.4492 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.16986386406208587 = (79667647.35580252 - 93200301.77639475) / 79667647.35580252 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 91.44 ns 344.87 ns 3.77 0.03 False
AnyWithPredicate_LastElementMatches - Duration of single invocation 658.20 ns 709.34 ns 1.08 0.25 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 344.86553527054144 > 96.14430677722734.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1574.8830110456286 (T) = (0 -344.54034905917706) / Math.Sqrt((0.08979140424076736 / (33)) + (0.6003643840692703 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.7647112346579874 = (91.5184001065828 - 344.54034905917706) / 91.5184001065828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 709.3373602705857 > 691.1506394736041.
IsChangePoint: Marked as a change because one of 7/28/2022 4:17:13 PM, 9/7/2022 1:05:52 PM, 9/24/2022 1:05:07 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -10.219784417914509 (T) = (0 -721.7666811879305) / Math.Sqrt((472.7199515181515 / (46)) + (145.69084668043394 / (13))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (13) - 2, .025) and -0.07023899405602117 = (674.397667433663 - 721.7666811879305) / 674.397667433663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 278.09 μs 2.77 ms 9.94 0.05 False
ReadLine - Duration of single invocation 146.63 μs 569.91 μs 3.89 0.04 False
ReadLine - Duration of single invocation 254.24 μs 1.39 ms 5.46 0.06 False
ReadLine - Duration of single invocation 58.77 μs 302.46 μs 5.15 0.05 False
ReadLine - Duration of single invocation 45.38 μs 250.05 μs 5.51 0.05 False
ReadLine - Duration of single invocation 37.55 μs 230.83 μs 6.15 0.09 False
ReadLine - Duration of single invocation 39.76 μs 235.31 μs 5.92 0.11 False
ReadLine - Duration of single invocation 29.22 μs 229.65 μs 7.86 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.765019944444445 > 291.9939420858253.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -94.09881089010355 (T) = (0 -2843354.3347866484) / Math.Sqrt((4948086.217916407 / (33)) + (19302346787.49349 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -9.184409734471503 = (279186.95426821394 - 2843354.3347866484) / 279186.95426821394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 569.907825595238 > 153.79726202832944.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -106.52486733305767 (T) = (0 -564786.6195933742) / Math.Sqrt((1340653.796818438 / (33)) + (400163476.97620463 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.859805592171662 = (146325.1467221191 - 564786.6195933742) / 146325.1467221191 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3872725375744048 > 266.3739204918033.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -117.12391621422546 (T) = (0 -1414968.0864497249) / Math.Sqrt((83985603.4058742 / (33)) + (2476017151.564777 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.509492159074204 = (256823.68639353703 - 1414968.0864497249) / 256823.68639353703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 302.461322676282 > 61.65940735238014.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -109.87058652266093 (T) = (0 -314025.997401401) / Math.Sqrt((583822.4465028116 / (33)) + (139723108.95663428 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.330865970654699 = (58907.126746395115 - 314025.997401401) / 58907.126746395115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.04736951058197 > 46.69311177436441.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -100.78161982383628 (T) = (0 -259507.58894058305) / Math.Sqrt((363165.92782993865 / (33)) + (117690170.70633216 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.789044595815213 = (44827.360481585514 - 259507.58894058305) / 44827.360481585514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 230.83270115546216 > 39.441336793979815.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -94.69843768326965 (T) = (0 -241120.49963993346) / Math.Sqrt((441172.46549482807 / (33)) + (119601688.8637828 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -5.392611889916743 = (37718.62014965432 - 241120.49963993346) / 37718.62014965432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 235.31414305703618 > 40.439667092049106.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -95.33686002084251 (T) = (0 -244995.52751505486) / Math.Sqrt((490470.6247357277 / (33)) + (121803630.98841389 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -5.393654325397782 = (38318.544457720964 - 244995.52751505486) / 38318.544457720964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.65053538359788 > 30.800908704485188.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -94.81752601599939 (T) = (0 -239939.39022753437) / Math.Sqrt((491097.5902657408 / (33)) + (127611214.12290281 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -7.117106861989384 = (29559.718050666237 - 239939.39022753437) / 29559.718050666237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 185.38 μs 875.01 μs 4.72 0.02 False
List - Duration of single invocation 181.90 μs 873.57 μs 4.80 0.01 False
Array - Duration of single invocation 187.58 μs 875.19 μs 4.67 0.02 False
Queue - Duration of single invocation 185.41 μs 878.07 μs 4.74 0.02 False
ImmutableArray - Duration of single invocation 187.88 μs 877.23 μs 4.67 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 875.0125159143518 > 194.7288293216912.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -118.01766779165715 (T) = (0 -875280.1168130791) / Math.Sqrt((1106683908.685404 / (33)) + (1107339.7210717571 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.5729464369556587 = (191403.9731013727 - 875280.1168130791) / 191403.9731013727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 873.5719053240741 > 192.7468898132695.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1563.570731822276 (T) = (0 -874909.0668199278) / Math.Sqrt((4774366.701293782 / (33)) + (1311664.5016363643 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.7489743934216206 = (184231.1611601592 - 874909.0668199278) / 184231.1611601592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 875.1912071314102 > 197.00813764322913.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1279.8210306099902 (T) = (0 -876964.6602057065) / Math.Sqrt((7482737.252184118 / (33)) + (1640465.4688213852 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.6660752368599474 = (187944.8177942075 - 876964.6602057065) / 187944.8177942075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 878.0666863425924 > 197.01721952380956.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -504.62645284498905 (T) = (0 -877793.1420847226) / Math.Sqrt((60032318.59714681 / (33)) + (1146939.1348322 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.645171609258209 = (188968.93719388294 - 877793.1420847226) / 188968.93719388294 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 877.2276000267094 > 197.25676574635597.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2261.631227811229 (T) = (0 -878256.0763528117) / Math.Sqrt((1865591.7175263297 / (33)) + (953837.6789951794 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.678066701355948 = (187739.10942703046 - 878256.0763528117) / 187739.10942703046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesWord - Duration of single invocation 7.41 μs 36.32 μs 4.90 0.06 False
MatchesWords - Duration of single invocation 15.06 μs 119.49 μs 7.94 0.01 False
ReplaceWords - Duration of single invocation 113.34 μs 257.22 μs 2.27 0.03 False
MatchesWord - Duration of single invocation 7.88 μs 56.83 μs 7.22 0.06 False
MatchWord - Duration of single invocation 384.60 ns 2.09 μs 5.45 0.04 False
MatchesWord - Duration of single invocation 7.70 μs 36.56 μs 4.75 0.05 False
MatchesWords - Duration of single invocation 114.26 μs 255.74 μs 2.24 0.03 False
MatchWord - Duration of single invocation 3.08 μs 6.78 μs 2.20 0.02 False
SplitWords - Duration of single invocation 116.04 μs 257.75 μs 2.22 0.03 False
Backtracking - Duration of single invocation 393.45 ns 959.50 ns 2.44 0.01 False
ReplaceWords - Duration of single invocation 13.03 μs 64.47 μs 4.95 0.04 False
SplitWords - Duration of single invocation 16.89 μs 121.46 μs 7.19 0.01 False
OneNodeBacktracking - Duration of single invocation 1.71 μs 2.05 μs 1.20 0.01 False
Backtracking - Duration of single invocation 377.49 ns 629.15 ns 1.67 0.07 False
MatchesWords - Duration of single invocation 11.03 μs 62.52 μs 5.67 0.02 False
OneNodeBacktracking - Duration of single invocation 243.21 ns 688.08 ns 2.83 0.05 False
ReplaceWords - Duration of single invocation 17.49 μs 121.86 μs 6.97 0.01 False
SplitWords - Duration of single invocation 12.88 μs 64.39 μs 5.00 0.03 False
Backtracking - Duration of single invocation 2.41 μs 2.73 μs 1.13 0.01 False
OneNodeBacktracking - Duration of single invocation 280.86 ns 1.26 μs 4.49 0.05 False
MatchWord - Duration of single invocation 523.80 ns 3.72 μs 7.10 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.31540372299383 > 7.716801481372267.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -169.51656219972205 (T) = (0 -35817.195963005324) / Math.Sqrt((1597.4238660077233 / (33)) + (732846.2681975941 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.8844088006218973 = (7332.964423134479 - 35817.195963005324) / 7332.964423134479 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 119.49086982530828 > 15.896871565769116.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -462.7977892359231 (T) = (0 -119029.52769404268) / Math.Sqrt((8530.493207968806 / (33)) + (1303289.0282913225 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -6.858216450200932 = (15147.142923379151 - 119029.52769404268) / 15147.142923379151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 257.21651475409834 > 119.0876856041797.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -249.93111899900276 (T) = (0 -257164.13086825874) / Math.Sqrt((4044961.924760459 / (33)) + (5310857.764631788 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.2503171246306501 = (114279.06229459446 - 257164.13086825874) / 114279.06229459446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.833460356060606 > 8.280705139720666.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -108.01939165895448 (T) = (0 -59609.54423827731) / Math.Sqrt((18487.82294473452 / (33)) + (5908449.195993811 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -6.402446542490472 = (8052.68148795335 - 59609.54423827731) / 8052.68148795335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0942115938024086 > 409.84759900501683.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -145.1487151375327 (T) = (0 -2140.5463230829905) / Math.Sqrt((248.04934359233806 / (33)) + (3574.4088091832905 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.4499963753079435 = (392.76105444419534 - 2140.5463230829905) / 392.76105444419534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.563674336615485 > 8.06436945447482.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -188.26328442051627 (T) = (0 -36306.450977546774) / Math.Sqrt((1133.2682683848425 / (33)) + (599773.9495657608 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.720446475814305 = (7691.3171589947315 - 36306.450977546774) / 7691.3171589947315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 255.74192775537628 > 117.41005810997596.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -195.8849899422147 (T) = (0 -256414.1871810293) / Math.Sqrt((3998893.598107371 / (33)) + (10968981.377764072 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.2881640210745546 = (112061.10436987533 - 256414.1871810293) / 112061.10436987533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.782885721655084 > 3.234091645950148.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -349.27522613665616 (T) = (0 -6811.651890907139) / Math.Sqrt((801.932201022169 / (33)) + (2333.218815098322 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.2103150205662554 = (3081.7561422362674 - 6811.651890907139) / 3081.7561422362674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 257.75159430555556 > 120.29777483921276.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -250.4020090300846 (T) = (0 -256424.33093189576) / Math.Sqrt((1902073.939715912 / (33)) + (6863349.818859775 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.2411015293765002 = (114418.88177339111 - 256424.33093189576) / 114418.88177339111 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 959.5031183884788 > 408.2113447612366.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -483.86911815551855 (T) = (0 -971.3940337130364) / Math.Sqrt((8.77121928237522 / (33)) + (30.777642956995383 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.49825591732117 = (388.82887336644154 - 971.3940337130364) / 388.82887336644154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.46888262419165 > 13.679182432862497.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -93.59518109350158 (T) = (0 -67626.5888109567) / Math.Sqrt((9739.137863189824 / (33)) + (8841391.977267558 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.192573475777929 = (13023.713410396214 - 67626.5888109567) / 13023.713410396214 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.46246947178477 > 17.806746697006552.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -809.2421389171116 (T) = (0 -120689.65217661488) / Math.Sqrt((12665.047252796563 / (33)) + (416496.20710050536 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -6.079812583612768 = (17047.012297467903 - 120689.65217661488) / 17047.012297467903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0510489618857117 > 1.8034764762183413.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 6:35:49 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -76.39780168300965 (T) = (0 -2044.5641209904065) / Math.Sqrt((24.9046851915706 / (32)) + (452.94420738021915 / (26))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (26) - 2, .025) and -0.18963580102316657 = (1718.6471012657355 - 2044.5641209904065) / 1718.6471012657355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 629.153457167329 > 393.9569869712741.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -98.34145049887404 (T) = (0 -626.9412987146611) / Math.Sqrt((20.29577552145013 / (33)) + (153.39328270601388 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.6676888853174272 = (375.93420705405094 - 626.9412987146611) / 375.93420705405094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.51617659718269 > 11.411699142174802.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -117.12554355936105 (T) = (0 -65053.12566460511) / Math.Sqrt((5566.855801832075 / (33)) + (5566485.9413534645 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -5.002737611259854 = (10837.242917727961 - 65053.12566460511) / 10837.242917727961 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 688.0847610445683 > 255.25037020413845.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -250.31060149706872 (T) = (0 -691.5895354483823) / Math.Sqrt((2.2168325979938404 / (33)) + (81.54314929907969 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8392835688215676 = (243.57888836563922 - 691.5895354483823) / 243.57888836563922 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.85705986757105 > 18.315714551576484.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -782.7010799518008 (T) = (0 -120856.69891503916) / Math.Sqrt((27344.18361115993 / (33)) + (432839.9302983595 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -5.951695831707685 = (17385.21101049824 - 120856.69891503916) / 17385.21101049824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.39307332084068 > 13.522751031183393.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -115.94194226792281 (T) = (0 -67090.15281393385) / Math.Sqrt((12597.678637776915 / (33)) + (5682558.583456617 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.225285087636524 = (12839.520081435354 - 67090.15281393385) / 12839.520081435354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7270721506770377 > 2.530135949623037.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -105.74916330032433 (T) = (0 -2686.763136079154) / Math.Sqrt((24.432373423464238 / (33)) + (156.59299715330656 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11403025604769128 = (2411.750597879753 - 2686.763136079154) / 2411.750597879753 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2620121688078525 > 293.3052705686421.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -93.26220325956048 (T) = (0 -1298.4545652555605) / Math.Sqrt((2517.7250668882666 / (33)) + (1066.1661903926004 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.5027003293204113 = (288.3724143932837 - 1298.4545652555605) / 288.3724143932837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7198923545358906 > 550.3983047700709.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -376.0082646522102 (T) = (0 -3706.4198148734163) / Math.Sqrt((15.917780425857158 / (33)) + (1849.707297175905 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -6.070558977787424 = (524.2046387728823 - 3706.4198148734163) / 524.2046387728823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyThreeValues - Duration of single invocation 459.31 ns 5.53 μs 12.05 0.03 False
IndexOfAnyFourValues - Duration of single invocation 617.73 ns 7.08 μs 11.46 0.06 False
IndexOfAnyTwoValues - Duration of single invocation 373.58 ns 3.47 μs 9.29 0.25 False
LastIndexOfValue - Duration of single invocation 199.98 ns 1.62 μs 8.08 0.15 False
IndexOfValue - Duration of single invocation 191.94 ns 1.97 μs 10.27 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.532557989967189 > 483.4885330930044.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -294.0898578724599 (T) = (0 -5601.284806424852) / Math.Sqrt((2.7364941708341863 / (33)) + (7938.725736879918 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -11.131778156924064 = (461.7035305107345 - 5601.284806424852) / 461.7035305107345 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.080614755320865 > 630.9245689089502.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -493.4405769445375 (T) = (0 -7114.397689852766) / Math.Sqrt((71.23077117860622 / (33)) + (4472.60773119649 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -10.816593057348248 = (602.0684350662832 - 7114.397689852766) / 602.0684350662832 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.472382871947531 > 392.50928657840245.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -108.20690853327785 (T) = (0 -3665.0956492207483) / Math.Sqrt((139.52302670338457 / (33)) + (23823.59560850841 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -8.592231198644404 = (382.0900031828578 - 3665.0956492207483) / 382.0900031828578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6156141255291958 > 209.97813173359592.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -115.55614331838746 (T) = (0 -1692.4399748849012) / Math.Sqrt((102.36525967309557 / (33)) + (4228.17820209579 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -7.262234445571739 = (204.8404685238614 - 1692.4399748849012) / 204.8404685238614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.971036866668168 > 201.33410838785232.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -151.80041585128967 (T) = (0 -1956.084385365915) / Math.Sqrt((0.06489636099172807 / (33)) + (3512.4866649102473 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -9.204658310698766 = (191.6854367691192 - 1956.084385365915) / 191.6854367691192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 24.27 μs 53.76 μs 2.22 0.08 False
Utf8JsonReaderCommentParsing - Duration of single invocation 44.27 μs 154.97 μs 3.50 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 43.78 μs 155.42 μs 3.55 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 5.16 μs 18.03 μs 3.50 0.14 False
Utf8JsonReaderCommentParsing - Duration of single invocation 3.45 μs 14.96 μs 4.33 0.18 False
Utf8JsonReaderCommentParsing - Duration of single invocation 24.20 μs 53.91 μs 2.23 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 5.05 μs 18.13 μs 3.59 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 3.52 μs 14.99 μs 4.26 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.7583199933687 > 25.47271618379554.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -155.5220068938099 (T) = (0 -54470.932120961355) / Math.Sqrt((771.4745173515457 / (33)) + (979979.0423580405 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.2445522059279082 = (24268.06201125664 - 54470.932120961355) / 24268.06201125664 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.97320827439887 > 46.49665613744703.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -175.20590492290373 (T) = (0 -156433.4093568751) / Math.Sqrt((126728.54084263276 / (33)) + (10608974.16413906 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5560896765371903 = (43990.288093410825 - 156433.4093568751) / 43990.288093410825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.4217221534653 > 46.33008464395341.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -193.8083693739125 (T) = (0 -156712.4464531689) / Math.Sqrt((112474.60246884637 / (33)) + (8697078.340329157 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5574933619957734 = (44051.36721470995 - 156712.4464531689) / 44051.36721470995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.030714897049155 > 5.32657784819661.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -352.6192668032452 (T) = (0 -18102.501499787308) / Math.Sqrt((8154.063824594193 / (33)) + (29043.13848383534 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.564337877147787 = (5078.783808866369 - 18102.501499787308) / 5078.783808866369 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.955383961323765 > 3.6972357900469714.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -197.2959706357869 (T) = (0 -15056.377669525666) / Math.Sqrt((5111.557443008041 / (33)) + (84889.95212730192 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.279158249169265 = (3518.537243264756 - 15056.377669525666) / 3518.537243264756 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.90672271759382 > 25.443750205023985.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -392.3139398966196 (T) = (0 -54265.097710059206) / Math.Sqrt((888.6470332382893 / (33)) + (151610.95940146552 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.238840638191656 = (24238.034983092817 - 54265.097710059206) / 24238.034983092817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.13447329924411 > 5.293053087609779.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -279.17836161603043 (T) = (0 -18181.154450881302) / Math.Sqrt((319.2565304424613 / (33)) + (57353.56230319844 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.607199086962771 = (5040.241476161404 - 18181.154450881302) / 5040.241476161404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.986089567130575 > 3.6881210497554493.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -189.92059668212642 (T) = (0 -15045.017468494372) / Math.Sqrt((24663.711336519245 / (33)) + (76631.70324902427 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.297613330386588 = (3500.7843451427984 - 15045.017468494372) / 3500.7843451427984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateElementsWithNamespace - Duration of single invocation 412.17 ns 471.77 ns 1.14 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Xml.Linq.Perf_XElement.CreateElementsWithNamespace

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 471.76948790873115 > 433.0392027842648.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -28.441574114896607 (T) = (0 -490.57453364130845) / Math.Sqrt((18.17554141397837 / (33)) + (180.45330823752002 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.18861076303186172 = (412.72933823177755 - 490.57453364130845) / 412.72933823177755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 97.14 μs 444.15 μs 4.57 0.09 False
ImmutableArray - Duration of single invocation 100.99 μs 448.61 μs 4.44 0.05 False
Array - Duration of single invocation 99.00 μs 446.51 μs 4.51 0.07 False
List - Duration of single invocation 97.89 μs 443.67 μs 4.53 0.08 False
Queue - Duration of single invocation 95.46 μs 444.35 μs 4.65 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 444.1544285714285 > 102.4900699112602.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2748.6204849947076 (T) = (0 -444705.43259457) / Math.Sqrt((151936.92569243672 / (33)) + (295076.6640929827 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.559319727943915 = (97537.67209370853 - 444705.43259457) / 97537.67209370853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 448.61431440476196 > 107.28027380548946.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -110.51978204177728 (T) = (0 -448314.7663517092) / Math.Sqrt((317703101.8821327 / (33)) + (214749.74677352648 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.259646682830416 = (105246.93706610818 - 448314.7663517092) / 105246.93706610818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 446.505724744898 > 104.46542050139468.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1683.2496743186118 (T) = (0 -446183.3864334403) / Math.Sqrt((840530.9506924306 / (32)) + (417900.3597382719 / (26))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (26) - 2, .025) and -3.469535507260026 = (99827.68583193683 - 446183.3864334403) / 99827.68583193683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 443.6657950148809 > 102.23618894156115.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2175.565457990446 (T) = (0 -444199.5092170661) / Math.Sqrt((523515.9220112742 / (33)) + (247767.09294251542 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.5551933573029397 = (97514.96245596693 - 444199.5092170661) / 97514.96245596693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 444.3485033275463 > 100.40131572217987.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1665.500809667389 (T) = (0 -443952.22480390116) / Math.Sqrt((1104435.6098087376 / (33)) + (261652.487278849 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.6024714551330277 = (96459.52813216728 - 443952.22480390116) / 96459.52813216728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments_DefineMessage - Duration of single invocation 245.39 ns 514.55 ns 2.10 0.06 False
FourArguments_DefineMessage - Duration of single invocation 478.68 ns 990.02 ns 2.07 0.05 False
FourArguments_EnumerableArgument - Duration of single invocation 2.66 μs 3.33 μs 1.25 0.20 False
TwoArguments - Duration of single invocation 384.02 ns 669.37 ns 1.74 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 514.54682516533 > 257.3558665693722.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -161.73434747253046 (T) = (0 -519.2275603804242) / Math.Sqrt((6.677510200050632 / (33)) + (69.74503971090127 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.1234195545177617 = (244.52424358423275 - 519.2275603804242) / 244.52424358423275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 990.0169019867699 > 446.3128905329113.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -142.4318113671904 (T) = (0 -1005.3910995046149) / Math.Sqrt((104.61646883488544 / (33)) + (348.0792002989232 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.361088010675951 = (425.81686703698244 - 1005.3910995046149) / 425.81686703698244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_EnumerableArgument

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.332980092775215 > 2.851254352682503.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -30.19558402824449 (T) = (0 -3391.1039624543655) / Math.Sqrt((7059.0868560344725 / (33)) + (6003.217589086596 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2312182180702716 = (2754.267206807054 - 3391.1039624543655) / 2754.267206807054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 669.3727801641236 > 404.5417736482337.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -105.81510311845983 (T) = (0 -669.6835156117072) / Math.Sqrt((64.4706138728576 / (33)) + (134.36857356693554 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.7291063872237697 = (387.300353847482 - 669.6835156117072) / 387.300353847482 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ILogB - Duration of single invocation 15.72 μs 17.52 μs 1.11 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.MathBenchmarks.Single.ILogB

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.517705652095664 > 16.509142062871653.
IsChangePoint: Marked as a change because one of 7/12/2022 12:16:14 PM, 7/18/2022 2:17:16 AM, 7/27/2022 9:27:06 AM, 7/28/2022 6:38:38 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 9/2/2022 11:09:40 AM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -150.72637838770748 (T) = (0 -17516.276564708136) / Math.Sqrt((4356.685359715239 / (33)) + (192.474813184232 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11309630877630936 = (15736.53279289443 - 17516.276564708136) / 15736.53279289443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PopCount_ulong - Duration of single invocation 747.42 ns 1.10 μs 1.48 0.03 False
Log2_uint - Duration of single invocation 795.81 ns 1.54 μs 1.93 0.03 False
PopCount_uint - Duration of single invocation 671.32 ns 1.10 μs 1.65 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_BitOperations.PopCount_ulong

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.10388389921774 > 783.4980728803803.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -719.8052261275213 (T) = (0 -1102.5325004889078) / Math.Sqrt((3.1129640852778415 / (33)) + (3.9406750750308253 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.47872700020580705 = (745.5957051811855 - 1102.5325004889078) / 745.5957051811855 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.Log2_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5357971286326169 > 835.113629125041.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2369.6189885048316 (T) = (0 -1536.1130954640118) / Math.Sqrt((1.1145448614620368 / (33)) + (1.6669965549521952 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9328888740686914 = (794.7239575291907 - 1536.1130954640118) / 794.7239575291907 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.PopCount_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.104743912912594 > 704.1827759274706.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -532.9060950877731 (T) = (0 -1105.2627877528494) / Math.Sqrt((18.394293300473702 / (33)) + (2.8348638169997535 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.649099415494932 = (670.2220480874618 - 1105.2627877528494) / 670.2220480874618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HtmlDecode_Entities - Duration of single invocation 784.36 ns 1.53 μs 1.95 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5259795918751173 > 823.263434855216.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -125.85560356763091 (T) = (0 -1560.3731773808283) / Math.Sqrt((2.0146951725833295 / (33)) + (989.1621203914884 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9916224063384993 = (783.4683785514837 - 1560.3731773808283) / 783.4683785514837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 35.57 μs 41.73 μs 1.17 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndRemove<Int32>.List(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.729990141025645 > 36.64756327204059.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -80.83839867113844 (T) = (0 -41342.71582734066) / Math.Sqrt((94901.67821840363 / (33)) + (88948.97357709461 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.18365763520252032 = (34927.93405608966 - 41342.71582734066) / 34927.93405608966 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 1.45 μs 1.67 μs 1.15 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndClear<String>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6726569240971387 > 1.5245743826109406.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -44.844528321661755 (T) = (0 -1668.0306065904942) / Math.Sqrt((387.14687008990967 / (33)) + (336.5476880822323 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.15413376477963733 = (1445.2662745803793 - 1668.0306065904942) / 1445.2662745803793 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 292.26 ns 344.27 ns 1.18 0.04 False
Span - Duration of single invocation 569.45 ns 683.07 ns 1.20 0.04 False
ConcurrentStack - Duration of single invocation 3.33 μs 3.73 μs 1.12 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<String>.Array(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 344.2699848087527 > 306.86728952120546.
IsChangePoint: Marked as a change because one of 7/13/2022 12:52:03 AM, 7/18/2022 2:17:16 AM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -8.898128319201314 (T) = (0 -392.4265913692007) / Math.Sqrt((0.15771392453451066 / (33)) + (3284.904014029793 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.34205234636330767 = (292.40781287898193 - 392.4265913692007) / 292.40781287898193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 683.0749141044197 > 597.3233580473321.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 7/27/2022 9:27:06 AM, 7/28/2022 6:38:38 AM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/24/2022 1:05:07 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -13.571522581300554 (T) = (0 -683.2136669673916) / Math.Sqrt((6828.9921703387245 / (46)) + (0.35652214274869326 / (13))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (13) - 2, .025) and -0.3193544931127272 = (517.8393453267431 - 683.2136669673916) / 517.8393453267431 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ConcurrentStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.727673072717875 > 3.495833405956835.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/1/2022 12:35:11 AM, 8/30/2022 6:17:03 PM, 9/6/2022 10:17:32 AM, 9/24/2022 1:05:07 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -35.59144793125293 (T) = (0 -3700.8091075957514) / Math.Sqrt((1571.4299366342786 / (46)) + (1065.472314144865 / (13))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (13) - 2, .025) and -0.11561630220561737 = (3317.2777237829046 - 3700.8091075957514) / 3317.2777237829046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf - Duration of single invocation 15.65 ns 23.73 ns 1.52 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.725407165026816 > 15.252189591147795.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -115.81390854971039 (T) = (0 -23.71208897957846) / Math.Sqrt((0.19391572301284885 / (33)) + (0.006463472258479288 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.618755674297861 = (14.648343388735075 - 23.71208897957846) / 14.648343388735075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileName - Duration of single invocation 62.65 ns 121.75 ns 1.94 0.01 False
GetFileNameWithoutExtension - Duration of single invocation 72.81 ns 140.31 ns 1.93 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.74813360454249 > 65.132065822826.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -111.89253616836498 (T) = (0 -123.44171083201576) / Math.Sqrt((0.16530142728804406 / (33)) + (7.716231390598071 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9918512379962597 = (61.97335849046351 - 123.44171083201576) / 61.97335849046351 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 140.3061060147931 > 75.02070844745299.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -197.75575230735538 (T) = (0 -142.38258397543987) / Math.Sqrt((1.6203048223043852 / (33)) + (2.0656391560636265 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9919192461150617 = (71.48009852966462 - 142.38258397543987) / 71.48009852966462 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetEnvironmentVariable - Duration of single invocation 115.37 ns 292.82 ns 2.54 0.05 False
ExpandEnvironmentVariables - Duration of single invocation 344.36 ns 803.52 ns 2.33 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Environment.GetEnvironmentVariable

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 292.81782768281994 > 121.39127517016294.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -106.73349437222384 (T) = (0 -284.88685423208767) / Math.Sqrt((12.37757128893488 / (33)) + (54.88593830306101 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.4433459803291557 = (116.59701758394 - 284.88685423208767) / 116.59701758394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.ExpandEnvironmentVariables

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 803.5157863185902 > 362.023613996252.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -107.2349430299102 (T) = (0 -791.51168828778) / Math.Sqrt((12.542801118036781 / (33)) + (439.83133556323077 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.2907218375443275 = (345.52937651141747 - 791.51168828778) / 345.52937651141747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyThreeValues - Duration of single invocation 438.59 ns 2.02 μs 4.61 0.11 False
IndexOfAnyTwoValues - Duration of single invocation 330.32 ns 1.41 μs 4.26 0.08 False
LastIndexOfAnyValues - Duration of single invocation 331.67 ns 1.25 μs 3.78 0.09 False
LastIndexOfValue - Duration of single invocation 182.04 ns 719.11 ns 3.95 0.06 False
IndexOfValue - Duration of single invocation 217.71 ns 757.49 ns 3.48 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0197895883192194 > 459.89053496405563.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1218.4666511448083 (T) = (0 -2016.0268385218187) / Math.Sqrt((34.98337491464623 / (33)) + (16.267675805849453 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.6452434265294693 = (433.998103739425 - 2016.0268385218187) / 433.998103739425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.405795156183465 > 343.61137215359895.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -112.6647494215698 (T) = (0 -1425.631846452063) / Math.Sqrt((18.8358168475837 / (33)) + (2448.4013529984704 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.333029320105146 = (329.01504724123777 - 1425.631846452063) / 329.01504724123777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.253306310575636 > 350.34036015317474.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -106.25756684141508 (T) = (0 -1292.7051341545214) / Math.Sqrt((3.1113187536371716 / (33)) + (2116.674404613693 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.8772060547753986 = (333.4115122827554 - 1292.7051341545214) / 333.4115122827554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 719.1058530419682 > 194.5240304061684.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -362.9943198478778 (T) = (0 -715.6884959762821) / Math.Sqrt((6.370670239690058 / (33)) + (50.673480359841804 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.8807172188101577 = (184.42170754088463 - 715.6884959762821) / 184.42170754088463 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 757.4887014038877 > 228.673851549639.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -64.73907602176477 (T) = (0 -800.2749808733503) / Math.Sqrt((0.03368000772781066 / (33)) + (2104.437380910335 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.6737528585876276 = (217.83582393210185 - 800.2749808733503) / 217.83582393210185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 505.50 ns 2.17 μs 4.29 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.166415056922935 > 527.5692069017953.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2679.728442403425 (T) = (0 -2165.3611632224565) / Math.Sqrt((6.252200951392747 / (33)) + (5.095325583067767 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.3160909487058845 = (501.69498023940105 - 2165.3611632224565) / 501.69498023940105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteFormat - Duration of single invocation 307.18 ns 636.46 ns 2.07 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_StreamWriter.WriteFormat

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 636.4630176301115 > 324.56125961067664.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -116.48491853370736 (T) = (0 -653.5488657550349) / Math.Sqrt((8.661689307221765 / (33)) + (221.64514180895324 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.120201867183073 = (308.2484153376151 - 653.5488657550349) / 308.2484153376151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.MathBenchmarks.MathTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DivRemInt64 - Duration of single invocation 6.81 ns 5.60 ns 0.82 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.MathTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.MathBenchmarks.MathTests.DivRemInt64

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.598749721675241 < 6.238007267080025.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 9/2/2022 1:41:42 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 24.479051891333143 (T) = (0 -5.649545227758353) / Math.Sqrt((0.1508549918787073 / (299)) + (0.0008848911212800268 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09136694602712708 = (6.217631202228991 - 5.649545227758353) / 6.217631202228991 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf - Duration of single invocation 20.94 ns 17.86 ns 0.85 0.07 False
Indexer - Duration of single invocation 37.27 ns 34.15 ns 0.92 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.857449606389935 < 19.637738572773582.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/2/2022 11:09:40 AM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 13.192596730730788 (T) = (0 -17.956072172274855) / Math.Sqrt((2.0460020408024837 / (299)) + (0.013848267303488412 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.05935260998439131 = (19.089057560641187 - 17.956072172274855) / 19.089057560641187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.14749053401273 < 35.395455432783415.
IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 9/2/2022 11:09:40 AM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 19.40619665405949 (T) = (0 -34.02904290979419) / Math.Sqrt((2.6090242799456567 / (299)) + (0.281534565844937 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07385634515464821 = (36.74272639213448 - 34.02904290979419) / 36.74272639213448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 929.61 μs 793.90 μs 0.85 0.45 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 793.903551923077 < 883.1797996297604.
IsChangePoint: Marked as a change because one of 8/30/2022 9:25:07 PM, 9/12/2022 7:28:58 AM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 4.1537291136414245 (T) = (0 -814824.0530053171) / Math.Sqrt((4241857764.6473756 / (299)) + (4352631761.475187 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06427961071763041 = (870798.6513259893 - 814824.0530053171) / 870798.6513259893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LeadingZeroCount_ulong - Duration of single invocation 1.11 μs 750.36 ns 0.68 0.03 False
LeadingZeroCount_uint - Duration of single invocation 1.10 μs 670.01 ns 0.61 0.03 False
Log2_ulong - Duration of single invocation 1.32 μs 896.94 ns 0.68 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 750.3581857197858 < 1.0505911151424332.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 24.672018560954573 (T) = (0 -748.2495398565084) / Math.Sqrt((28366.248130205906 / (299)) + (2.247171633897561 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.24317409382686342 = (988.6679799849951 - 748.2495398565084) / 988.6679799849951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 670.0056521645673 < 1.045950804034561.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 24.34497121681058 (T) = (0 -669.8322581909213) / Math.Sqrt((41407.93544057749 / (299)) + (0.2780870008425182 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.29958574639309393 = (956.3372743223065 - 669.8322581909213) / 956.3372743223065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.Log2_ulong

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 896.9447046213169 < 1.2551643096693834.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/18/2022 2:17:16 AM, 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 23.960593606469704 (T) = (0 -897.3060652585987) / Math.Sqrt((41909.57069928215 / (299)) + (0.4675261399875009 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2402134996750829 = (1180.9976419360864 - 897.3060652585987) / 1180.9976419360864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 3.86 μs 3.50 μs 0.91 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateFor<String>.List(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4951613283884027 < 3.665199913203416.
IsChangePoint: Marked as a change because one of 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 12.633480430126264 (T) = (0 -3408.560341355015) / Math.Sqrt((51599.308419089655 / (299)) + (7178.89708434594 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07279448830418758 = (3676.164882929707 - 3408.560341355015) / 3676.164882929707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StructWithInterfaceInterfaceMethod - Duration of single invocation 174.75 μs 155.58 μs 0.89 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 155.5784565441176 < 166.37230543940143.
IsChangePoint: Marked as a change because one of 7/12/2022 3:04:36 AM, 7/28/2022 6:38:38 AM, 8/31/2022 4:31:58 AM, 9/2/2022 11:09:40 AM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 20.350640086820473 (T) = (0 -153619.04111673165) / Math.Sqrt((212239870.89071983 / (299)) + (1472687.770636565 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10392631156310354 = (171435.7235337458 - 153619.04111673165) / 171435.7235337458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 1.77 μs 1.35 μs 0.76 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSet<String>.Array(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3512738876140167 < 1.684117797598273.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 9/2/2022 11:09:40 AM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 25.9933066454151 (T) = (0 -1356.5217674952255) / Math.Sqrt((22366.197713296104 / (299)) + (55.364627090085236 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14388697534675515 = (1584.5124749090967 - 1356.5217674952255) / 1584.5124749090967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AndArray - Duration of single invocation 84.26 μs 78.07 μs 0.93 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'IfStatements.IfStatements*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

IfStatements.IfStatements.AndArray

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.06578669154229 < 79.78484491604596.
IsChangePoint: Marked as a change because one of 9/2/2022 11:09:40 AM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 19.558504796008776 (T) = (0 -79131.42492532749) / Math.Sqrt((3166971.2671751007 / (119)) + (1364056.0542383695 / (26))) is greater than 1.9766921979294603 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (119) + (26) - 2, .975) and 0.06498736909152006 = (84631.3967421398 - 79131.42492532749) / 84631.3967421398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 2.44 μs 975.69 ns 0.40 0.04 False
IndexOf_Word_NotFound - Duration of single invocation 2.40 μs 1.06 μs 0.44 0.07 False
IndexOf_Word_NotFound - Duration of single invocation 3.93 μs 1.44 μs 0.37 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 975.6938087447803 < 2.320662868446736.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.302661730176996 (T) = (0 -987.9908978624027) / Math.Sqrt((302342.6840119816 / (299)) + (84.64879543372236 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19055763038565934 = (1220.5821377167736 - 987.9908978624027) / 1220.5821377167736 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0641578341874867 < 2.28265359064261.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.549469841949701 (T) = (0 -1079.7062141380934) / Math.Sqrt((248331.8936715509 / (299)) + (753.6466126380008 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17012024594579486 = (1301.039348006037 - 1079.7062141380934) / 1301.039348006037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.437310257509735 < 3.736100171786583.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.483221509399381 (T) = (0 -1458.3464100005594) / Math.Sqrt((861314.3282472737 / (299)) + (191.60955671726813 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2161522195756866 = (1860.4969567064738 - 1458.3464100005594) / 1860.4969567064738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 406.34 μs 221.52 μs 0.55 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 51.96 μs 24.43 μs 0.47 0.02 False
Utf8JsonReaderCommentParsing - Duration of single invocation 400.81 μs 226.96 μs 0.57 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 52.52 μs 25.95 μs 0.49 0.03 False
Utf8JsonReaderCommentParsing - Duration of single invocation 227.33 μs 157.79 μs 0.69 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 228.23 μs 157.52 μs 0.69 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 35.85 μs 11.81 μs 0.33 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 36.21 μs 11.81 μs 0.33 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongMultiLine)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 221.5200247652582 < 380.8987628883928.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.810767721727008 (T) = (0 -226064.74045242654) / Math.Sqrt((4123338212.467376 / (299)) + (6900184.683079259 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11468037058229967 = (255348.16233670962 - 226064.74045242654) / 255348.16233670962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.434243947641267 < 48.876736464967.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.905550556709143 (T) = (0 -24307.418158754084) / Math.Sqrt((92669025.08129154 / (299)) + (34565.94624975269 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16971178119405406 = (29275.879879050997 - 24307.418158754084) / 29275.879879050997 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 226.96010790441179 < 380.55402635579554.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.796070486140423 (T) = (0 -229073.08379365856) / Math.Sqrt((4229344627.6577697 / (299)) + (3931806.3272301615 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11400946598122189 = (258550.2609769457 - 229073.08379365856) / 258550.2609769457 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.954098135125655 < 49.923938369605644.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.890098882444828 (T) = (0 -25596.14759762436) / Math.Sqrt((97624419.518174 / (299)) + (31119.544261139417 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16584964068298297 = (30685.292299798195 - 25596.14759762436) / 30685.292299798195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.7863530011655 < 216.14444497444356.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.734829336713918 (T) = (0 -159400.23297058998) / Math.Sqrt((693722237.9091736 / (299)) + (664752.1111318552 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06917762283125856 = (171246.6705575274 - 159400.23297058998) / 171246.6705575274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.51962607142858 < 216.81166562686633.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.77729401340063 (T) = (0 -159341.30613477767) / Math.Sqrt((667717200.9758719 / (299)) + (537625.374294353 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06827328389024169 = (171017.21285837542 - 159341.30613477767) / 171017.21285837542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.808846860431235 < 34.088118940485124.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.293984043845784 (T) = (0 -11824.047161067692) / Math.Sqrt((79120707.50503045 / (299)) + (27580.63639119746 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.26554605412225546 = (16099.099511184184 - 11824.047161067692) / 16099.099511184184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.812017867258309 < 34.02380613744088.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.348894304734616 (T) = (0 -11836.301751701303) / Math.Sqrt((79727457.32107018 / (299)) + (4798.023962940011 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2670559916154698 = (16148.984937866537 - 11836.301751701303) / 16148.984937866537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesWord - Duration of single invocation 91.97 μs 55.28 μs 0.60 0.04 False
MatchesWords - Duration of single invocation 165.96 μs 123.49 μs 0.74 0.02 False
ReplaceWords - Duration of single invocation 175.20 μs 131.11 μs 0.75 0.03 False
SplitWords - Duration of single invocation 131.52 μs 93.54 μs 0.71 0.02 False
MatchesWord - Duration of single invocation 59.87 μs 40.83 μs 0.68 0.04 False
MatchesWord - Duration of single invocation 60.11 μs 39.30 μs 0.65 0.04 False
MatchesWords - Duration of single invocation 129.94 μs 89.99 μs 0.69 0.02 False
ReplaceWords - Duration of single invocation 137.13 μs 99.95 μs 0.73 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 55.27998988409624 < 87.46834087530635.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.301179360102606 (T) = (0 -55841.12352052101) / Math.Sqrt((199010762.91101882 / (299)) + (519510.5109474676 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0854479612308305 = (61058.442989940304 - 55841.12352052101) / 61058.442989940304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 123.48768914041995 < 158.82561378641083.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.307788267374023 (T) = (0 -123160.1326922066) / Math.Sqrt((299602428.7387881 / (299)) + (3605994.346876319 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.05186384393428235 = (129897.09537421128 - 123160.1326922066) / 129897.09537421128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.105993847539 < 166.42462537609887.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.902514218989349 (T) = (0 -131081.55125505183) / Math.Sqrt((292024566.317127 / (299)) + (1374664.8413175887 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.05072032888222312 = (138085.2821810703 - 131081.55125505183) / 138085.2821810703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 93.54143993263473 < 125.54298471133474.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.009628908089944 (T) = (0 -94460.88577754221) / Math.Sqrt((202786031.60786605 / (299)) + (1499130.679978823 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06780600017616886 = (101331.7890861706 - 94460.88577754221) / 101331.7890861706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.83227419072615 < 57.00664619651721.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.15622249615946 (T) = (0 -39952.573863366284) / Math.Sqrt((58670952.77122654 / (299)) + (533528.8486012971 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07697117897456342 = (43284.210582916654 - 39952.573863366284) / 43284.210582916654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.29600288098237 < 57.11466761070464.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.139060891388722 (T) = (0 -39610.316930535366) / Math.Sqrt((58709004.608930334 / (299)) + (193242.7410223937 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08488416212102601 = (43284.4840958527 - 39610.316930535366) / 43284.4840958527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 89.99343635057471 < 123.19645363410964.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.9107733893763434 (T) = (0 -90327.70283402974) / Math.Sqrt((207877113.81314945 / (299)) + (964721.6082606213 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06972185183543456 = (97097.52186724571 - 90327.70283402974) / 97097.52186724571 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.95304303797471 < 131.07676948104353.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.086690288386556 (T) = (0 -98839.33364052759) / Math.Sqrt((197508086.93468127 / (299)) + (574069.4887502544 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06331832961631827 = (105520.730004294 - 98839.33364052759) / 105520.730004294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 59.89 ms 23.86 ms 0.40 0.10 False
Count - Duration of single invocation 59.90 ms 23.87 ms 0.40 0.12 False
Count - Duration of single invocation 105.40 ms 68.96 ms 0.65 0.04 False
Count - Duration of single invocation 96.20 ms 59.70 ms 0.62 0.01 False
Count - Duration of single invocation 96.65 ms 60.18 ms 0.62 0.02 False
Count - Duration of single invocation 59.70 ms 23.62 ms 0.40 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w.+-]+@[\w.-]+.[\w.-]+", Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.860457693333334 < 56.91597516791666.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.48698343978729 (T) = (0 -24339996.189980283) / Math.Sqrt((188378063632486.44 / (299)) + (3585297328202.6704 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1893320474229363 = (30024618.72657594 - 24339996.189980283) / 30024618.72657594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.867574835714287 < 56.93247724583333.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 5.860641004101149 (T) = (0 -24538277.07984925) / Math.Sqrt((189835950893042 / (299)) + (6822796202807.745 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18450064382417578 = (30089879.156886447 - 24538277.07984925) / 30089879.156886447 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.96034808928572 < 99.75755196105769.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 14.554166279923418 (T) = (0 -69525955.68715836) / Math.Sqrt((165746177674566.28 / (299)) + (718373573063.5476 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1377036001885454 = (80628836.7925003 - 69525955.68715836) / 80628836.7925003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.69503941666667 < 90.82689352362638.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.663628757472805 (T) = (0 -60474273.430107065) / Math.Sqrt((171894476817435.38 / (299)) + (1394907802252.4546 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0912967236022128 = (66550077.4574453 - 60474273.430107065) / 66550077.4574453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 60.18254413333333 < 91.52576406458333.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.933628329227416 (T) = (0 -60394256.34261763) / Math.Sqrt((174803162298318.38 / (299)) + (1201079614971.733 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09447754949790474 = (66695482.04920833 - 60394256.34261763) / 66695482.04920833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.617753614285714 < 56.68151644615384.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.903635018123445 (T) = (0 -23627986.22173993) / Math.Sqrt((181938033874267.66 / (299)) + (1539618566.493612 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20694342152274764 = (29793569.416078757 - 23627986.22173993) / 29793569.416078757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 566.95 ns 253.35 ns 0.45 0.11 False
IndexOfString - Duration of single invocation 296.65 ns 223.09 ns 0.75 0.23 False
IndexOfString - Duration of single invocation 4.83 μs 1.89 μs 0.39 0.11 False
IndexOfString - Duration of single invocation 2.10 μs 755.64 ns 0.36 0.11 False
IndexOfString - Duration of single invocation 537.82 ns 251.18 ns 0.47 0.11 False
IndexOfString - Duration of single invocation 1.33 μs 490.55 ns 0.37 0.06 False
IndexOfString - Duration of single invocation 4.83 μs 1.92 μs 0.40 0.10 False
IndexOfString - Duration of single invocation 234.29 ns 147.14 ns 0.63 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.3476375676629 < 538.4544259540471.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.852517896216977 (T) = (0 -255.99169373360198) / Math.Sqrt((13525.436219608562 / (299)) + (1.955163003504052 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17114439052343422 = (308.849564154201 - 255.99169373360198) / 308.849564154201 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 223.08843168317296 < 283.4379441543932.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.408557014110061 (T) = (0 -217.25663233239382) / Math.Sqrt((962.3003376341585 / (299)) + (41.841289213620584 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0608671944527122 = (231.33749673006645 - 217.25663233239382) / 231.33749673006645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.885460504189031 < 4.589858496965799.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.3347011805374684 (T) = (0 -1952.2512273902407) / Math.Sqrt((1247862.2966371598 / (299)) + (34533.21618882212 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19398932242152822 = (2422.115837541337 - 1952.2512273902407) / 2422.115837541337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 755.6358868690292 < 1.8906871973690977.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.7417910800561645 (T) = (0 -761.2284069927377) / Math.Sqrt((182079.83029128023 / (299)) + (183.60214284504846 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2015464029707216 = (953.3783927143162 - 761.2284069927377) / 953.3783927143162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 251.18069475302684 < 517.9226179855111.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.89930108242347 (T) = (0 -254.62403622253396) / Math.Sqrt((11481.198115406285 / (299)) + (26.76892916360691 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1630410828761565 = (304.22525050277665 - 254.62403622253396) / 304.22525050277665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 490.550344484023 < 1.2546348580478093.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.712350198686686 (T) = (0 -486.49335803591055) / Math.Sqrt((94264.32377729472 / (299)) + (20.442320420924272 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.21986596528134805 = (623.60227394945 - 486.49335803591055) / 623.60227394945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.922757431871309 < 4.587235314632722.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.199232963815231 (T) = (0 -1954.2399545268286) / Math.Sqrt((1216250.2740353209 / (299)) + (34554.655930721296 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18899482791444214 = (2409.6516542568534 - 1954.2399545268286) / 2409.6516542568534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.13664329074885 < 214.61953866901564.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.973382580975528 (T) = (0 -147.96880538818533) / Math.Sqrt((830.9631289343673 / (299)) + (11.30652880713415 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09805902478109012 = (164.05597423076588 - 147.96880538818533) / 164.05597423076588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetEnvironmentVariable - Duration of single invocation 845.81 ns 637.06 ns 0.75 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Environment.GetEnvironmentVariable

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 637.0601531110788 < 804.4181899305371.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.260098293799257 (T) = (0 -640.3550509327694) / Math.Sqrt((6549.8162360178 / (299)) + (204.49060649220857 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.05063917306716453 = (674.5117691464139 - 640.3550509327694) / 674.5117691464139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfShort - Duration of single invocation 705.38 ns 328.99 ns 0.47 0.09 False
IndexOfChar - Duration of single invocation 705.53 ns 318.24 ns 0.45 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Array.IndexOfShort

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 328.9909081125293 < 674.0020137982822.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.1732142551803735 (T) = (0 -322.1856328114663) / Math.Sqrt((21459.307278389606 / (299)) + (48.30519997336784 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1603992698024375 = (383.7367229726614 - 322.1856328114663) / 383.7367229726614 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfChar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 318.24430045357735 < 670.3100754103855.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.201786028680944 (T) = (0 -321.5212897860492) / Math.Sqrt((21032.131554850606 / (299)) + (37.939943207550165 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1595222782940748 = (382.5458801376133 - 321.5212897860492) / 382.5458801376133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in BenchmarksGame.ReverseComplement_6

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 14.60 ms 7.38 ms 0.51 0.57 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.ReverseComplement_6*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

BenchmarksGame.ReverseComplement_6.RunBench

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.380660884259259 < 13.994288100840336.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.209820062978599 (T) = (0 -7569383.054850235) / Math.Sqrt((6748004097922.853 / (299)) + (301679519030.8335 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16701106986124248 = (9087015.182289809 - 7569383.054850235) / 9087015.182289809 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 296.14 μs 107.49 μs 0.36 0.05 False
ReadLine - Duration of single invocation 274.51 μs 98.56 μs 0.36 0.07 False
ReadLineAsync - Duration of single invocation 289.06 μs 110.47 μs 0.38 0.08 False
ReadLine - Duration of single invocation 283.07 μs 105.52 μs 0.37 0.06 False
ReadLineAsync - Duration of single invocation 288.52 μs 114.82 μs 0.40 0.05 False
ReadLineAsync - Duration of single invocation 373.86 μs 189.62 μs 0.51 0.04 False
ReadLineAsync - Duration of single invocation 276.85 μs 98.00 μs 0.35 0.06 False
ReadLineAsync - Duration of single invocation 618.76 μs 441.14 μs 0.71 0.03 False
ReadLine - Duration of single invocation 348.54 μs 167.53 μs 0.48 0.04 False
ReadLine - Duration of single invocation 518.32 μs 335.07 μs 0.65 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 107.49230659246575 < 270.24664139507234.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.697139902292407 (T) = (0 -106672.91972375644) / Math.Sqrt((4329419687.487869 / (299)) + (224611.95573208638 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.21547261109639548 = (135970.93133081606 - 106672.91972375644) / 135970.93133081606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.55920965709903 < 260.9990757530664.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.685030344816834 (T) = (0 -95946.48654952027) / Math.Sqrt((4451357717.268508 / (299)) + (358192.0316471603 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.23616999915555487 = (125612.35673310491 - 95946.48654952027) / 125612.35673310491 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 110.47259909456739 < 274.315828986014.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.802304519648206 (T) = (0 -110099.46469956862) / Math.Sqrt((4493968439.771384 / (299)) + (609647.0307758487 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2156562491555389 = (140371.44374648284 - 110099.46469956862) / 140371.44374648284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 105.5176475950783 < 270.1983832916667.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.715838121550722 (T) = (0 -106130.10119499612) / Math.Sqrt((4487033982.369203 / (299)) + (878707.337032751 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.21994034932079068 = (136053.8275535558 - 106130.10119499612) / 136053.8275535558 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 114.8242725297619 < 273.2406776521916.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.654970676110571 (T) = (0 -111996.54520157186) / Math.Sqrt((4350757401.869082 / (299)) + (683480.822498542 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20695484689162755 = (141223.4155427303 - 111996.54520157186) / 141223.4155427303 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 189.61811955514364 < 359.2326324864211.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.336129825160965 (T) = (0 -189786.3107712052) / Math.Sqrt((4917517552.819868 / (299)) + (6264423.936677312 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13637175356109946 = (219754.63580976342 - 189786.3107712052) / 219754.63580976342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.00245165648496 < 263.05556526041664.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.4314162783103574 (T) = (0 -98400.30407845325) / Math.Sqrt((4428710075.25775 / (299)) + (1658998.8408605687 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.225575008041056 = (127062.40772207662 - 98400.30407845325) / 127062.40772207662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 441.14196296296296 < 587.7474399851191.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.722100602330896 (T) = (0 -437756.4423841759) / Math.Sqrt((4732577334.724503 / (299)) + (35347452.64108036 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.05985158820716008 = (465624.82783902716 - 437756.4423841759) / 465624.82783902716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 167.53449046391756 < 331.18729276938336.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.462485421533708 (T) = (0 -162462.6421992903) / Math.Sqrt((4961052182.927012 / (299)) + (4256246.991928393 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1582661392030016 = (193009.51258567884 - 162462.6421992903) / 193009.51258567884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 335.0697590016366 < 495.42439349486364.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.142418862793207 (T) = (0 -338196.47328832134) / Math.Sqrt((4781620482.351631 / (299)) + (14022761.577812815 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07907765625428272 = (367236.6900261715 - 338196.47328832134) / 367236.6900261715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyThreeValues - Duration of single invocation 5.10 μs 2.59 μs 0.51 0.13 False
IndexOfValue - Duration of single invocation 2.59 μs 976.38 ns 0.38 0.08 False
IndexOfAnyTwoValues - Duration of single invocation 4.13 μs 1.36 μs 0.33 0.08 False
LastIndexOfValue - Duration of single invocation 2.46 μs 845.12 ns 0.34 0.04 False
IndexOfAnyFourValues - Duration of single invocation 6.30 μs 2.89 μs 0.46 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.587916913789791 < 4.824054390369061.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.417953287320444 (T) = (0 -2508.517205951607) / Math.Sqrt((996589.1483758874 / (299)) + (1845.1910869129226 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14714332892799073 = (2941.311583807504 - 2508.517205951607) / 2941.311583807504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 976.3798718730407 < 2.380714373505079.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.4282858959324995 (T) = (0 -980.7928367022834) / Math.Sqrt((335805.15757425054 / (299)) + (147.21705274930528 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20284123912190313 = (1230.3607321857787 - 980.7928367022834) / 1230.3607321857787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3629983525134934 < 3.963421428199415.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.622716373885821 (T) = (0 -1394.2603661938956) / Math.Sqrt((1064004.016315393 / (299)) + (514.0128941660787 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2464449574677041 = (1850.2435621803172 - 1394.2603661938956) / 1850.2435621803172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 845.1227413139693 < 2.3623644539185684.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.492981553495436 (T) = (0 -836.4673928532079) / Math.Sqrt((389480.77837660065 / (299)) + (105.95527541505193 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2446050533955342 = (1107.3245811521065 - 836.4673928532079) / 1107.3245811521065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.8907862448502737 < 5.859503537934527.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.011681737003963 (T) = (0 -2975.8338790862467) / Math.Sqrt((1631948.1692271584 / (299)) + (2347.544251635317 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13079467562982078 = (3423.6259208864344 - 2975.8338790862467) / 3423.6259208864344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Get - Duration of single invocation 109.81 ms 102.03 ms 0.93 0.01 False
Get - Duration of single invocation 523.16 ms 487.70 ms 0.93 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 10)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.03297364285714 < 104.66932205597527.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 22.231358524343158 (T) = (0 -103654992.5061637) / Math.Sqrt((1094454063766.9471 / (48)) + (1407358481904.2979 / (26))) is greater than 1.9934635666612894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (26) - 2, .975) and 0.0561472796285236 = (109821151.40312117 - 103654992.5061637) / 109821151.40312117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 40)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 487.69964246666666 < 498.4694506233333.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 29.469775162629627 (T) = (0 -489518298.84322345) / Math.Sqrt((33200383800096.73 / (48)) + (21909684581402.797 / (26))) is greater than 1.9934635666612894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (26) - 2, .975) and 0.06939618910310306 = (526022237.5098977 - 489518298.84322345) / 526022237.5098977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddChainedConfigurationEmpty - Duration of single invocation 397.31 ms 311.17 ms 0.78 0.01 False
AddChainedConfigurationNoDelimiter - Duration of single invocation 9.87 ms 8.86 ms 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 311.1655152 < 376.4560290021795.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 122.01219033133296 (T) = (0 -312127427.1096224) / Math.Sqrt((15666353361177.998 / (48)) + (4260785857520.536 / (26))) is greater than 1.9934635666612894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (26) - 2, .975) and 0.214889326759125 = (397558507.0333906 - 312127427.1096224) / 397558507.0333906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.863978930864198 < 9.257947782083333.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 31.251150144075503 (T) = (0 -8879067.939764552) / Math.Sqrt((11253835138.56798 / (48)) + (10288014098.981134 / (26))) is greater than 1.9934635666612894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (26) - 2, .975) and 0.08118020846900893 = (9663557.556775885 - 8879067.939764552) / 9663557.556775885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyTwoValues - Duration of single invocation 3.56 μs 1.27 μs 0.36 0.12 False
IndexOfValue - Duration of single invocation 2.21 μs 808.78 ns 0.37 0.11 False
IndexOfAnyThreeValues - Duration of single invocation 4.60 μs 1.50 μs 0.33 0.07 False
LastIndexOfAnyValues - Duration of single invocation 3.58 μs 1.31 μs 0.37 0.10 False
LastIndexOfValue - Duration of single invocation 2.18 μs 812.91 ns 0.37 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2749380387740816 < 3.3859693669076596.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.83848469722137 (T) = (0 -1264.5470557948756) / Math.Sqrt((716100.7898313216 / (299)) + (561.8472717574278 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.23355125790576736 = (1649.8781801632904 - 1264.5470557948756) / 1649.8781801632904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 808.7792305326898 < 2.096576892235065.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.217810403921542 (T) = (0 -802.8370819010478) / Math.Sqrt((267750.76138162345 / (299)) + (51.791634695174515 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2346832841503163 = (1049.0259330213473 - 802.8370819010478) / 1049.0259330213473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.499961405112011 < 4.338387975381167.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.108204151255425 (T) = (0 -1494.2616136774636) / Math.Sqrt((1321833.0974264329 / (299)) + (557.1176165448318 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.26560238385796464 = (2034.6765578123386 - 1494.2616136774636) / 2034.6765578123386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3103845197794242 < 3.408318162425592.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.596115304601404 (T) = (0 -1324.0741247778542) / Math.Sqrt((746792.9380011625 / (299)) + (9215.46983652977 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.21013920124515018 = (1676.3385736640525 - 1324.0741247778542) / 1676.3385736640525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 812.9055888342326 < 2.089257472348765.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.735581235013536 (T) = (0 -821.115934732283) / Math.Sqrt((257969.9302462748 / (299)) + (186.1198260483364 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2174438651309097 = (1049.2741646829504 - 821.115934732283) / 1049.2741646829504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileNameWithoutExtension - Duration of single invocation 579.26 ns 483.95 ns 0.84 0.08 False
GetFileName - Duration of single invocation 474.53 ns 415.66 ns 0.88 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 483.9464430283123 < 552.0451649763613.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 22.943346512475078 (T) = (0 -486.52753792894265) / Math.Sqrt((544.5861698884036 / (299)) + (82.96229824407882 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09549391963394861 = (537.8930539992016 - 486.52753792894265) / 537.8930539992016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFileName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 415.6613047214568 < 456.4276518968219.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 22.027614285044 (T) = (0 -413.1894068901765) / Math.Sqrt((229.7493963160094 / (299)) + (110.9052746003765 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10683328053574183 = (462.6117362926565 - 413.1894068901765) / 462.6117362926565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf - Duration of single invocation 153.60 ns 115.64 ns 0.75 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.64484346234015 < 145.56926013162428.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.319319588430647 (T) = (0 -115.08372156711981) / Math.Sqrt((239.28177399277175 / (299)) + (5.528668467415274 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06015860994203483 = (122.4501525305477 - 115.08372156711981) / 122.4501525305477 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_String - Duration of single invocation 375.86 ns 217.57 ns 0.58 0.16 False
IndexOfAny - Duration of single invocation 1.19 μs 421.35 ns 0.35 0.07 False
Replace_Char - Duration of single invocation 92.11 ns 64.04 ns 0.70 0.28 False
Replace_String - Duration of single invocation 1.58 μs 1.19 μs 0.75 0.02 False
Replace_Char - Duration of single invocation 325.68 ns 157.70 ns 0.48 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 217.56606859316392 < 356.95416555445775.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.02535398151502 (T) = (0 -219.5627303940504) / Math.Sqrt((3512.3850964439393 / (299)) + (29.017668446461247 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11589942415295908 = (248.3458742051958 - 219.5627303940504) / 248.3458742051958 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.IndexOfAny

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 421.35102658929884 < 1.1322141869081872.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.391475816524924 (T) = (0 -416.90930765824476) / Math.Sqrt((83155.81290669972 / (299)) + (11.84691821543583 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2283398981776024 = (540.2758373455455 - 416.90930765824476) / 540.2758373455455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 64.03807158177555 < 87.06650021874385.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.024442856357472 (T) = (0 -66.3525140913186) / Math.Sqrt((98.78516767482861 / (299)) + (15.62465573716229 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08056273721602666 = (72.16644003573354 - 66.3525140913186) / 72.16644003573354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.187637755003849 < 1.5137682879899614.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.252145789223064 (T) = (0 -1162.1697513334727) / Math.Sqrt((26795.80700007726 / (299)) + (288.9745646146624 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.058938585647194154 = (1234.9563308073036 - 1162.1697513334727) / 1234.9563308073036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.69783942035002 < 308.79114231169973.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.765515210785861 (T) = (0 -154.43247856238838) / Math.Sqrt((4105.237660824519 / (299)) + (10.885909700074631 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15905756785388667 = (183.64215273127735 - 154.43247856238838) / 183.64215273127735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 104.15 secs 45.82 secs 0.44 0.06 False
Count - Duration of single invocation 61.94 secs 29.85 secs 0.48 0.07 False
Count - Duration of single invocation 109.32 secs 46.51 secs 0.43 0.07 False
Count - Duration of single invocation 60.72 secs 29.65 secs 0.49 0.07 False
Count - Duration of single invocation 61.18 secs 29.64 secs 0.48 0.08 False
Count - Duration of single invocation 104.51 secs 46.95 secs 0.45 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.822741626 < 98.98967971969168.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.99234581265566 (T) = (0 -46251779073.16667) / Math.Sqrt((4.7895416679307293E+20 / (299)) + (3.716465916328915E+17 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16120882297145106 = (55140993777.51616 - 46251779073.16667) / 55140993777.51616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.852982254 < 58.8485040549.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.579445073794037 (T) = (0 -29950164986.44871) / Math.Sqrt((1.4495948161519087E+20 / (299)) + (1.0145325358228174E+17 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15032154753220228 = (35248822539.2344 - 29950164986.44871) / 35248822539.2344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.506653396 < 98.34021078396457.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.941440518190577 (T) = (0 -46264583770.10257) / Math.Sqrt((4.7905325081833806E+20 / (299)) + (4.464790349990545E+17 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16031949196924952 = (55097841771.51375 - 46264583770.10257) / 55097841771.51375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.65255915 < 58.633671685699994.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.6423067639367686 (T) = (0 -30026480554.679485) / Math.Sqrt((1.4725103047961767E+20 / (299)) + (5.914325211277622E+17 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13702818275746573 = (34794277118.60106 - 30026480554.679485) / 34794277118.60106 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.639091680666667 < 58.10870191670833.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.353196672711986 (T) = (0 -29689958489.435894) / Math.Sqrt((1.4179317649514943E+20 / (299)) + (1.2977946077799974E+17 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.14635559510763715 = (34780241420.52397 - 29689958489.435894) / 34780241420.52397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.94696885 < 99.32857285018125.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.881236740164076 (T) = (0 -46793966501.628204) / Math.Sqrt((4.950614808733961E+20 / (299)) + (1.7782382887842554E+17 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15938976188729242 = (55666662598.2185 - 46793966501.628204) / 55666662598.2185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 7.45 ms 3.50 ms 0.47 0.05 False
Count - Duration of single invocation 5.84 ms 2.39 ms 0.41 0.07 False
Count - Duration of single invocation 5.85 ms 2.37 ms 0.41 0.09 False
Count - Duration of single invocation 6.48 ms 2.35 ms 0.36 0.07 False
Count - Duration of single invocation 6.23 ms 2.70 ms 0.43 0.05 False
Count - Duration of single invocation 7.97 ms 4.66 ms 0.58 0.05 False
Count - Duration of single invocation 12.49 ms 6.62 ms 0.53 0.07 False
Count - Duration of single invocation 5.62 ms 2.17 ms 0.39 0.07 False
Count - Duration of single invocation 14.77 ms 8.27 ms 0.56 0.05 False
Count - Duration of single invocation 12.62 ms 6.70 ms 0.53 0.05 False
Count - Duration of single invocation 5.97 ms 2.33 ms 0.39 0.07 False
Count - Duration of single invocation 6.22 ms 2.75 ms 0.44 0.02 False
Count - Duration of single invocation 24.66 ms 21.92 ms 0.89 0.04 False
Count - Duration of single invocation 14.29 ms 7.57 ms 0.53 0.04 False
Count - Duration of single invocation 13.43 ms 7.52 ms 0.56 0.08 False
Count - Duration of single invocation 27.98 ms 20.16 ms 0.72 0.04 False
Count - Duration of single invocation 6.10 ms 2.63 ms 0.43 0.06 False
Count - Duration of single invocation 6.64 ms 3.12 ms 0.47 0.04 False
Count - Duration of single invocation 5.86 ms 2.37 ms 0.41 0.07 False
Count - Duration of single invocation 10.73 ms 4.31 ms 0.40 0.07 False
Count - Duration of single invocation 19.29 ms 15.76 ms 0.82 0.03 False
Count - Duration of single invocation 5.85 ms 2.38 ms 0.41 0.05 False
Count - Duration of single invocation 12.57 ms 6.04 ms 0.48 0.03 False
Count - Duration of single invocation 5.62 ms 2.17 ms 0.39 0.09 False
Count - Duration of single invocation 12.57 ms 6.57 ms 0.52 0.06 False
Count - Duration of single invocation 12.57 ms 6.73 ms 0.54 0.06 False
Count - Duration of single invocation 14.71 ms 8.32 ms 0.57 0.08 False
Count - Duration of single invocation 8.23 ms 4.57 ms 0.55 0.07 False
Count - Duration of single invocation 12.56 ms 6.75 ms 0.54 0.08 False
Count - Duration of single invocation 6.42 ms 3.10 ms 0.48 0.04 False
Count - Duration of single invocation 13.15 ms 6.35 ms 0.48 0.03 False
Count - Duration of single invocation 5.79 ms 2.34 ms 0.40 0.06 False
Count - Duration of single invocation 15.73 ms 9.80 ms 0.62 0.04 False
Count - Duration of single invocation 15.17 ms 8.48 ms 0.56 0.06 False
Count - Duration of single invocation 7.93 ms 4.34 ms 0.55 0.08 False
Count - Duration of single invocation 6.78 ms 2.70 ms 0.40 0.08 False
Count - Duration of single invocation 5.80 ms 2.34 ms 0.40 0.10 False
Count - Duration of single invocation 6.14 ms 2.64 ms 0.43 0.06 False
Count - Duration of single invocation 6.66 ms 3.21 ms 0.48 0.03 False
Count - Duration of single invocation 5.82 ms 2.17 ms 0.37 0.08 False
Count - Duration of single invocation 5.92 ms 2.46 ms 0.42 0.08 False
Count - Duration of single invocation 6.12 ms 2.62 ms 0.43 0.04 False
Count - Duration of single invocation 11.34 ms 5.11 ms 0.45 0.03 False
Count - Duration of single invocation 12.42 ms 6.67 ms 0.54 0.06 False
Count - Duration of single invocation 6.10 ms 2.44 ms 0.40 0.05 False
Count - Duration of single invocation 7.93 ms 4.32 ms 0.54 0.10 False
Count - Duration of single invocation 6.41 ms 2.94 ms 0.46 0.05 False
Count - Duration of single invocation 6.24 ms 2.78 ms 0.45 0.05 False
Count - Duration of single invocation 6.20 ms 2.70 ms 0.44 0.08 False
Count - Duration of single invocation 11.03 ms 4.61 ms 0.42 0.03 False
Count - Duration of single invocation 13.67 ms 7.64 ms 0.56 0.04 False
Count - Duration of single invocation 7.92 ms 4.32 ms 0.55 0.08 False
Count - Duration of single invocation 12.01 ms 5.13 ms 0.43 0.05 False
Count - Duration of single invocation 12.09 ms 5.50 ms 0.46 0.04 False
Count - Duration of single invocation 7.93 ms 4.34 ms 0.55 0.10 False
Count - Duration of single invocation 5.92 ms 2.46 ms 0.41 0.10 False
Count - Duration of single invocation 11.26 ms 4.50 ms 0.40 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.499480676056338 < 6.612926073909747.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.139897101968216 (T) = (0 -3452470.060574111) / Math.Sqrt((1739287912143.3706 / (299)) + (1132841925.4309509 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15289605109185175 = (4075615.5900631547 - 3452470.060574111) / 4075615.5900631547 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3871307946428573 < 5.554742240052084.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.713485381363699 (T) = (0 -2422519.536098688) / Math.Sqrt((1697811819334.6895 / (299)) + (27311746841.93253 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1852201762275276 = (2973219.8385599414 - 2422519.536098688) / 2973219.8385599414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3742320267857147 < 5.558026415451389.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.626296920252028 (T) = (0 -2374918.9364922335) / Math.Sqrt((1714387260025.9321 / (299)) + (779540905.8750234 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19600591330744593 = (2953900.9997724006 - 2374918.9364922335) / 2953900.9997724006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3542518443452387 < 5.516420496493056.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.562763755469803 (T) = (0 -2335292.473974183) / Math.Sqrt((1717023400478.5164 / (299)) + (438595904.64104086 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1972835512050623 = (2909237.1004481036 - 2335292.473974183) / 2909237.1004481036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.703353327380952 < 5.908142296316964.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.220108055646741 (T) = (0 -2745317.3747461876) / Math.Sqrt((1750122955089.4414 / (299)) + (12287914392.87141 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1729902961794319 = (3319570.9337672107 - 2745317.3747461876) / 3319570.9337672107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.655044274553572 < 7.663026097643229.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.2081268187406415 (T) = (0 -4420317.393040514) / Math.Sqrt((1835463504476.953 / (299)) + (5889359213.943992 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11512191684071209 = (4995397.080305816 - 4420317.393040514) / 4995397.080305816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.623695809829061 < 11.93900494025641.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.594311841005195 (T) = (0 -6445190.3811774) / Math.Sqrt((5248504290345.793 / (299)) + (11436963555.016844 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13648347139881964 = (7463887.682170986 - 6445190.3811774) / 7463887.682170986 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.170116847355769 < 5.341407143787201.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.587370862769574 (T) = (0 -2175893.8759139897) / Math.Sqrt((1702160570255.5603 / (299)) + (666878910.9895616 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.2086669766526494 = (2749656.3541730205 - 2175893.8759139897) / 2749656.3541730205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.26799786 < 14.026070820119045.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 5.9946828350108055 (T) = (0 -8324308.818247644) / Math.Sqrt((6323270715333.903 / (299)) + (15709464777.666191 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0960134981719152 = (9208443.711730018 - 8324308.818247644) / 9208443.711730018 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.702962927350427 < 11.925266963596865.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.788582554199806 (T) = (0 -6463712.4560800595) / Math.Sqrt((5317004379605.8955 / (299)) + (80259327008.67102 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13173616682655817 = (7444410.568681245 - 6463712.4560800595) / 7444410.568681245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3289008717261908 < 5.592269942093063.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.713061064634135 (T) = (0 -2326883.1321928524) / Math.Sqrt((1687300897417.2583 / (299)) + (496291442.20267326 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19963409707699986 = (2907274.1900859214 - 2326883.1321928524) / 2907274.1900859214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7483510877976194 < 5.900082176021825.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.646838506624388 (T) = (0 -2727184.0907721943) / Math.Sqrt((1708806282367.33 / (299)) + (617366929.822976 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1751977263181885 = (3306470.141744875 - 2727184.0907721943) / 3306470.141744875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.922394248484846 < 23.668822842678573.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 15.929012982644043 (T) = (0 -21600768.495947752) / Math.Sqrt((5803214652330.46 / (299)) + (90983165633.66792 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10040621972930648 = (24011691.68760587 - 21600768.495947752) / 24011691.68760587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.565460910714285 < 13.480195504509803.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.825799916462891 (T) = (0 -7560146.605601976) / Math.Sqrt((6025883668695.931 / (299)) + (15000314553.88521 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12970813284223762 = (8686909.404648622 - 7560146.605601976) / 8686909.404648622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.51716713131313 < 12.773638870986677.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.749380901101179 (T) = (0 -7300309.199932621) / Math.Sqrt((5313524144657.784 / (299)) + (14603374264.2411 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12566548795288418 = (8349560.836664337 - 7300309.199932621) / 8349560.836664337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.15800222222222 < 26.768725063148146.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.161917837959646 (T) = (0 -20784797.445362546) / Math.Sqrt((7443333307178.667 / (299)) + (78364009623.40619 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.061562579654437004 = (22148304.186025437 - 20784797.445362546) / 22148304.186025437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.630584044097222 < 5.809922541473214.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.921859274634028 (T) = (0 -2648341.2176823583) / Math.Sqrt((1677142440928.8562 / (299)) + (3159507441.064388 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18463317496187362 = (3248036.511123104 - 2648341.2176823583) / 3248036.511123104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.123770452678572 < 6.313514663130629.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.308388562175354 (T) = (0 -3129738.3859922662) / Math.Sqrt((1690456697466.455 / (299)) + (940621343.0239825 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16683632276686716 = (3756450.8289486007 - 3129738.3859922662) / 3756450.8289486007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.373820950595238 < 5.563001356319445.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.4277008841965495 (T) = (0 -2435299.884237896) / Math.Sqrt((1748530169906.7715 / (299)) + (30736396561.489456 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18120118938932206 = (2974234.7603333676 - 2435299.884237896) / 2974234.7603333676 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.3093343950892855 < 10.243880618787879.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.516515420421528 (T) = (0 -4345259.267822772) / Math.Sqrt((5734145977695.955 / (299)) + (5623205454.649889 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1941325735100686 = (5392027.429063808 - 4345259.267822772) / 5392027.429063808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.763952704761905 < 18.372941547103178.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 10.74345062827263 (T) = (0 -15800733.260370942) / Math.Sqrt((1932288924063.2517 / (299)) + (42268333123.29177 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.05762569708499005 = (16766939.857650135 - 15800733.260370942) / 16766939.857650135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.375022573511905 < 5.540069855868055.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.712548192875869 (T) = (0 -2374638.3703043237) / Math.Sqrt((1685928112294.7314 / (299)) + (121789771.43662648 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1961323432755814 = (2954016.5603632387 - 2374638.3703043237) / 2954016.5603632387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.037297196864112 < 11.918831640705129.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.075086324530329 (T) = (0 -5990993.666904354) / Math.Sqrt((5839813056183.1455 / (299)) + (3059761093.442683 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15891243371622935 = (7122913.12707752 - 5990993.666904354) / 7122913.12707752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1653524657451926 < 5.341763713658234.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.637372015484659 (T) = (0 -2174790.062955509) / Math.Sqrt((1763395621942.9526 / (299)) + (703530372.8236113 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.21278972559084816 = (2762654.571026551 - 2174790.062955509) / 2762654.571026551 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.571059876068378 < 11.902861760833334.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.456964221725481 (T) = (0 -6432625.524337057) / Math.Sqrt((5371805283782.703 / (299)) + (6969430332.077123 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13534901967420265 = (7439563.096213997 - 6432625.524337057) / 7439563.096213997 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.731283831481482 < 11.941826422499998.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.107627810733639 (T) = (0 -6551953.93614842) / Math.Sqrt((5281291193926.156 / (299)) + (147849443647.04807 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12468312195527695 = (7485236.6045815665 - 6551953.93614842) / 7485236.6045815665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.319470702917773 < 14.103683203377976.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 5.756302982562092 (T) = (0 -8352460.724122049) / Math.Sqrt((7018494319346.389 / (299)) + (16679367368.101063 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.09667506780173957 = (9246352.476728566 - 8352460.724122049) / 9246352.476728566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.56523051875 < 7.572380880572916.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.890690108807733 (T) = (0 -4417830.244628329) / Math.Sqrt((1761822491249.4546 / (299)) + (8642203257.296242 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12351315290208088 = (5040383.959275522 - 4417830.244628329) / 5040383.959275522 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.754044313513514 < 11.932010399880951.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.3845069492468625 (T) = (0 -6457966.44052496) / Math.Sqrt((5235215020614.783 / (299)) + (14195217987.629602 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13318398283528504 = (7450215.8620100785 - 6457966.44052496) / 7450215.8620100785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.100413259375 < 6.085878334945436.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.360136172277492 (T) = (0 -2943418.84887324) / Math.Sqrt((1720427008729.7656 / (299)) + (1975939295.9202619 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16031743108361157 = (3505394.7263329835 - 2943418.84887324) / 3505394.7263329835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.349714015779093 < 12.269506753333332.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.737882168250471 (T) = (0 -6422711.903400201) / Math.Sqrt((5882511152914.453 / (299)) + (6287516111.49863 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1453144117723736 = (7514707.152976652 - 6422711.903400201) / 7514707.152976652 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.335531463988095 < 5.504149288767361.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.690609429059196 (T) = (0 -2327972.040856328) / Math.Sqrt((1712486110938.0403 / (299)) + (281127066.3332671 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20015864282796747 = (2910542.2218816574 - 2327972.040856328) / 2910542.2218816574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.803633675595238 < 14.946358187039683.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.952290655394455 (T) = (0 -9556805.787235843) / Math.Sqrt((5294526662627.072 / (299)) + (15484933418.894672 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1011839642418345 = (10632660.530110065 - 9556805.787235843) / 10632660.530110065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.479674177011493 < 14.58047919478621.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.623980925158953 (T) = (0 -8565199.538242899) / Math.Sqrt((6357386250512.383 / (299)) + (12481331310.18609 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10236113757932627 = (9541921.475129787 - 8565199.538242899) / 9541921.475129787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.335851582331731 < 7.545819001111779.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.989323490171917 (T) = (0 -4429112.544492232) / Math.Sqrt((1821169961376.5903 / (299)) + (9740183484.166216 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11260005631663596 = (4991112.041441144 - 4429112.544492232) / 4991112.041441144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7031197127976188 < 5.925169146153846.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.575364022503992 (T) = (0 -2709009.5108983163) / Math.Sqrt((1748983608686.9355 / (299)) + (1301174476.4558582 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17680779654862558 = (3290859.04791169 - 2709009.5108983163) / 3290859.04791169 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.336914051020408 < 5.494449548392856.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 5.513035912811409 (T) = (0 -2397902.0016663144) / Math.Sqrt((1701023827444.7236 / (299)) + (79149718715.84982 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1768572321628713 = (2913105.836994702 - 2397902.0016663144) / 2913105.836994702 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6399535574468085 < 5.8365848246041665.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.914329369309927 (T) = (0 -2676132.0934460433) / Math.Sqrt((1674017972444.036 / (299)) + (3718565121.422425 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18306867483768569 = (3275834.8358282484 - 2676132.0934460433) / 3275834.8358282484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.207310684166667 < 6.327987440602678.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.831841160314503 (T) = (0 -3180650.8084736895) / Math.Sqrt((1720289988932.7495 / (299)) + (669741614.5458298 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.15767500043362928 = (3776037.5272146617 - 3180650.8084736895) / 3776037.5272146617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1705517018229163 < 5.344502812021902.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.024581651689166 (T) = (0 -2220313.036498971) / Math.Sqrt((1748840210706.4827 / (299)) + (64305962819.20807 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19841817573192688 = (2769914.3983539618 - 2220313.036498971) / 2769914.3983539618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4648357187499994 < 5.644730155447155.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.764421220444003 (T) = (0 -2492531.447958639) / Math.Sqrt((1710522211329.667 / (299)) + (3633467309.371996 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19255360561394497 = (3086931.176222348 - 2492531.447958639) / 3086931.176222348 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6222067847222226 < 5.817784904047618.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.667257129843607 (T) = (0 -2651701.8437043806) / Math.Sqrt((1768697980356.915 / (299)) + (1000732209.3713642 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18241063529020665 = (3243317.4380217297 - 2651701.8437043806) / 3243317.4380217297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.112407781333333 < 10.799831453476191.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.658946768269421 (T) = (0 -4974371.644821858) / Math.Sqrt((5742828591830.094 / (299)) + (5410838626.572793 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17663967841027575 = (6041548.899535821 - 4974371.644821858) / 6041548.899535821 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.66730129009009 < 11.87352828015873.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.744910832197163 (T) = (0 -6435503.521882244) / Math.Sqrt((5288566981699.75 / (299)) + (9512844088.123041 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13919347176947727 = (7476132.337322175 - 6435503.521882244) / 7476132.337322175 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4402730806547623 < 5.619833214525746.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.630952500616571 (T) = (0 -2456651.0938963564) / Math.Sqrt((1702684826432.0032 / (299)) + (2033295984.8135898 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1909444124193194 = (3036442.8991121384 - 2456651.0938963564) / 3036442.8991121384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.318506388822115 < 7.548404644363839.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.139559186085672 (T) = (0 -4404506.914470781) / Math.Sqrt((1800404887296.9043 / (299)) + (8275985491.185485 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12826891992556044 = (5052598.23258185 - 4404506.914470781) / 5052598.23258185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.938196271875 < 6.098114673055554.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 4.586085745524962 (T) = (0 -3033593.0575533295) / Math.Sqrt((1689296484029.2043 / (299)) + (111938682833.27649 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13106747025861232 = (3491172.16092277 - 3033593.0575533295) / 3491172.16092277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.77880922 < 5.918835392291667.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.064520181653762 (T) = (0 -2784626.131832331) / Math.Sqrt((1688492175528.0928 / (299)) + (1086363001.095525 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.17927644413334118 = (3392891.6892020376 - 2784626.131832331) / 3392891.6892020376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7003943816964293 < 5.883746329444445.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 5.814066613475362 (T) = (0 -2753377.013591215) / Math.Sqrt((1791885134945.9666 / (299)) + (66453826088.102325 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.16334769485278092 = (3290945.350478327 - 2753377.013591215) / 3290945.350478327 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.613325743131869 < 10.39023760151515.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.672180293384281 (T) = (0 -4618280.897364635) / Math.Sqrt((5432610209608.076 / (299)) + (859677202.7490475 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20213475273864537 = (5788296.849896304 - 4618280.897364635) / 5788296.849896304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.6364898214285715 < 12.828142622519257.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.588063839094009 (T) = (0 -7394102.879253283) / Math.Sqrt((5991699687033.844 / (299)) + (34942638680.24265 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.130484236063233 = (8503701.929193543 - 7394102.879253283) / 8503701.929193543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.324892987980769 < 7.53189838015625.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.678706825230666 (T) = (0 -4373178.084148813) / Math.Sqrt((1718595019032.2334 / (299)) + (2992653004.55786 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.13191041022977823 = (5037703.637600778 - 4373178.084148813) / 5037703.637600778 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.126573023049645 < 11.107563263257575.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 8.863214854974352 (T) = (0 -5129223.776171461) / Math.Sqrt((5883558855029.723 / (299)) + (4116795478.1824703 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.19573322014117595 = (6377515.402379062 - 5129223.776171461) / 6377515.402379062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.501087838518518 < 11.384171477738095.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.782037973086556 (T) = (0 -5531661.044518643) / Math.Sqrt((5778418023636.154 / (299)) + (3419468278.938012 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.1640447846871899 = (6617173.914572354 - 5531661.044518643) / 6617173.914572354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.338035423437501 < 7.562971779244792.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 6.903323725203767 (T) = (0 -4463312.442287826) / Math.Sqrt((1764743785788.2769 / (299)) + (30093856222.655674 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.11500860283346696 = (5043339.920114436 - 4463312.442287826) / 5043339.920114436 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.455248654879774 < 5.646362176369605.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.277603235435141 (T) = (0 -2507560.508221014) / Math.Sqrt((1700779428336.6284 / (299)) + (12614245201.544134 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.18569000684862336 = (3079368.458339513 - 2507560.508221014) / 3079368.458339513 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.504719423030303 < 10.557364330339226.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsImprovementStdDev: Marked as improvement because 7.735599804508538 (T) = (0 -4517691.769726755) / Math.Sqrt((6835606995309.618 / (299)) + (2740957736.8406734 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.20603113178992286 = (5690011.221613054 - 4517691.769726755) / 5690011.221613054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 33.93 ms 116.82 ms 3.44 0.01 False
Count - Duration of single invocation 55.02 ms 195.39 ms 3.55 0.01 False
Count - Duration of single invocation 12.44 ms 60.39 ms 4.85 0.01 False
Count - Duration of single invocation 63.95 ms 176.57 ms 2.76 0.01 False
Count - Duration of single invocation 31.08 ms 117.44 ms 3.78 0.02 False
Count - Duration of single invocation 25.20 ms 99.56 ms 3.95 0.01 False
Count - Duration of single invocation 34.69 ms 115.76 ms 3.34 0.01 False
Count - Duration of single invocation 39.42 ms 125.98 ms 3.20 0.01 False
Count - Duration of single invocation 27.69 ms 85.03 ms 3.07 0.01 False
Count - Duration of single invocation 40.67 ms 180.61 ms 4.44 0.01 False
Count - Duration of single invocation 27.42 ms 84.88 ms 3.10 0.01 False
Count - Duration of single invocation 24.99 ms 97.97 ms 3.92 0.01 False
Count - Duration of single invocation 12.42 ms 60.28 ms 4.86 0.01 False
Count - Duration of single invocation 29.46 ms 103.64 ms 3.52 0.01 False
Count - Duration of single invocation 12.62 ms 60.43 ms 4.79 0.01 False
Count - Duration of single invocation 39.80 ms 179.77 ms 4.52 0.01 False
Count - Duration of single invocation 42.92 ms 156.14 ms 3.64 0.01 False
Count - Duration of single invocation 112.49 ms 208.30 ms 1.85 0.01 False
Count - Duration of single invocation 27.39 ms 84.69 ms 3.09 0.01 False
Count - Duration of single invocation 28.83 ms 110.15 ms 3.82 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 116.82069535714285 > 36.159513878571424.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1006.9214839592181 (T) = (0 -116137543.88999721) / Math.Sqrt((58112003628.146095 / (33)) + (125635812563.44218 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.378298526188745 = (34377525.54716315 - 116137543.88999721) / 34377525.54716315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 195.3882087142857 > 58.24394345875.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1176.3440427152675 (T) = (0 -195863362.0224007) / Math.Sqrt((155363668959.0086 / (33)) + (249110106812.66663 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.545265336567799 = (55246460.681562886 - 195863362.0224007) / 55246460.681562886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.39013239285714 > 13.075605576085529.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -4074.608084230184 (T) = (0 -60308582.50927373) / Math.Sqrt((2849554127.135446 / (33)) + (1339883513.2063644 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.839057144062242 = (12462878.762090107 - 60308582.50927373) / 12462878.762090107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\s", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 176.569206 > 67.03737070625.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -483.0422007354112 (T) = (0 -176994229.0550366) / Math.Sqrt((327513514131.50964 / (33)) + (1157980601565.7927 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.7540953550128913 = (64265831.87574787 - 176994229.0550366) / 64265831.87574787 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.44140039285715 > 33.171034242908654.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -679.2121692163156 (T) = (0 -117396856.58624259) / Math.Sqrt((433505237423.31616 / (33)) + (77291665736.61363 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.7639667956558798 = (31189663.18239955 - 117396856.58624259) / 31189663.18239955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.56466148333334 > 26.507990076000006.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1268.2439108760614 (T) = (0 -99322625.03295118) / Math.Sqrt((80637652751.799 / (33)) + (24958056853.945732 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.9205294114059583 = (25333982.891186282 - 99322625.03295118) / 25333982.891186282 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.75665939285715 > 35.968405792857155.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -695.193108547825 (T) = (0 -116174162.45815018) / Math.Sqrt((281111121898.3982 / (33)) + (140804800969.7199 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.4057019907989687 = (34111664.136208236 - 116174162.45815018) / 34111664.136208236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.97555632142857 > 41.570367457500005.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -717.4153088108133 (T) = (0 -125152134.57136871) / Math.Sqrt((49456353906.361565 / (33)) + (330216142946.17346 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.155280397301993 = (39664346.369464785 - 125152134.57136871) / 39664346.369464785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.026573 > 29.09500295194444.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3556.0168894365793 (T) = (0 -85085753.33542898) / Math.Sqrt((1523906052.3933086 / (33)) + (5564668215.554483 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.069013295314807 = (27724139.698359057 - 85085753.33542898) / 27724139.698359057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 180.6096682142857 > 42.76783187163462.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3644.6120926021385 (T) = (0 -180820677.18450972) / Math.Sqrt((16951178691.348131 / (33)) + (25015136000.804832 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.430892565428871 = (40809086.321642265 - 180820677.18450972) / 40809086.321642265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.87504373333333 > 28.827984869444442.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3440.095647120706 (T) = (0 -84865117.93157932) / Math.Sqrt((3111126155.577564 / (33)) + (4788046812.636913 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.0901965851602857 = (27462692.289260115 - 84865117.93157932) / 27462692.289260115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.97306946153846 > 25.957535838250006.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1078.6544279471636 (T) = (0 -98080946.26363762) / Math.Sqrt((134000178542.41875 / (33)) + (14916157989.754349 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.97882678481349 = (24650720.31735486 - 98080946.26363762) / 24650720.31735486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.27843946428572 > 13.03424264061298.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2965.440784263855 (T) = (0 -60272595.67313857) / Math.Sqrt((5910714578.140285 / (33)) + (2110291126.8928826 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.848657784101133 = (12430779.476904696 - 60272595.67313857) / 12430779.476904696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 103.64415339285715 > 30.921161801249998.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1113.7618070715823 (T) = (0 -103646537.5703719) / Math.Sqrt((140000642851.63568 / (33)) + (5269263488.806423 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5265667213931464 = (29390210.29763107 - 103646537.5703719) / 29390210.29763107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.4273105 > 13.305684537598685.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3401.0713037633923 (T) = (0 -60415470.74128452) / Math.Sqrt((4666113934.642713 / (33)) + (1455299779.2698605 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.7818624837759427 = (12634296.980781877 - 60415470.74128452) / 12634296.980781877 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 179.76939442857145 > 41.97483748541668.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1278.4930081939822 (T) = (0 -180038811.33587632) / Math.Sqrt((359484771519.6417 / (33)) + (28046777279.50722 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.484239653761468 = (40149239.388857424 - 180038811.33587632) / 40149239.388857424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\s", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.14148192857144 > 44.79430510916667.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -630.7499288449244 (T) = (0 -154803145.29979572) / Math.Sqrt((163492587734.6662 / (33)) + (689443541683.8811 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.6078757673630597 = (42907005.474010244 - 154803145.29979572) / 42907005.474010244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 208.2989643846154 > 118.10355667355769.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -456.3011649650477 (T) = (0 -206960635.30181742) / Math.Sqrt((336951200322.488 / (33)) + (850194486018.6859 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.8406544951227937 = (112438611.2929959 - 206960635.30181742) / 112438611.2929959 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.69462771153846 > 28.815475333675213.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2269.8223688880885 (T) = (0 -84779749.40489398) / Math.Sqrt((2029643685.3976629 / (33)) + (14985711000.556858 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.0882272120758594 = (27452562.128000394 - 84779749.40489398) / 27452562.128000394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.1472361 > 29.154254677393162.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -595.2328924566334 (T) = (0 -109918072.29240632) / Math.Sqrt((576539941068.9933 / (33)) + (39819572876.26623 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.9445744514471843 = (27865635.100911736 - 109918072.29240632) / 27865635.100911736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 5.14 μs 15.01 μs 2.92 0.43 False
Utf8JsonReaderCommentParsing - Duration of single invocation 5.04 μs 15.19 μs 3.01 0.38 False
Utf8JsonReaderCommentParsing - Duration of single invocation 24.77 μs 46.16 μs 1.86 0.14 False
Utf8JsonReaderCommentParsing - Duration of single invocation 24.91 μs 48.49 μs 1.95 0.29 False
Utf8JsonReaderCommentParsing - Duration of single invocation 3.54 μs 12.37 μs 3.50 0.21 False
Utf8JsonReaderCommentParsing - Duration of single invocation 43.93 μs 135.18 μs 3.08 0.44 False
Utf8JsonReaderCommentParsing - Duration of single invocation 3.61 μs 12.45 μs 3.45 0.21 False
Utf8JsonReaderCommentParsing - Duration of single invocation 43.82 μs 134.64 μs 3.07 0.38 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.011408981748321 > 5.394173264982537.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -587.1808333061666 (T) = (0 -15101.675936812415) / Math.Sqrt((4433.7898669507895 / (33)) + (4129.202941597781 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9917364271434868 = (5047.796256313766 - 15101.675936812415) / 5047.796256313766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.188629047792585 > 5.294666737551537.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -555.2684264493922 (T) = (0 -15154.529464401918) / Math.Sqrt((6056.615869906122 / (33)) + (3762.5050195673243 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9747269302892159 = (5094.427091809913 - 15154.529464401918) / 5094.427091809913 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.16383683628319 > 26.23948329631979.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -46.68554627804696 (T) = (0 -45915.09662639252) / Math.Sqrt((1829793.7132978402 / (33)) + (3515076.7088841046 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.7984124606660824 = (25530.904411876036 - 45915.09662639252) / 25530.904411876036 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.48732187649306 > 26.178553132475255.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -42.55805062239388 (T) = (0 -46362.371792906975) / Math.Sqrt((2842867.4437229177 / (33)) + (4075958.4016528795 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.8262226218588875 = (25387.031809799475 - 46362.371792906975) / 25387.031809799475 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.367889872711347 > 3.7105345496591204.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -778.8293289689962 (T) = (0 -12442.056818163297) / Math.Sqrt((2060.366003858169 / (33)) + (1800.091131504208 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.54960528382913 = (3505.1944718601085 - 12442.056818163297) / 3505.1944718601085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.1830245535714 > 46.046411834107765.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1109.376515641662 (T) = (0 -134708.52955793898) / Math.Sqrt((20103.496310143844 / (33)) + (158465.02624380577 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.0702793033572227 = (43875.008182689045 - 134708.52955793898) / 43875.008182689045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.454767002721049 > 3.7136878272886142.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -641.8299623131534 (T) = (0 -12439.925438172677) / Math.Sqrt((4344.316812683595 / (33)) + (1588.9285108176302 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5251399719459164 = (3528.916734419967 - 12439.925438172677) / 3528.916734419967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.64488128698227 > 45.922405311794954.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1270.3391514497312 (T) = (0 -134370.8505872289) / Math.Sqrt((26221.383734359286 / (33)) + (111858.8349229539 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.0763608306702217 = (43678.507815988094 - 134370.8505872289) / 43678.507815988094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 11.64 secs 39.37 secs 3.38 0.01 False
Count - Duration of single invocation 7.08 secs 24.98 secs 3.53 0.01 False
Count - Duration of single invocation 9.56 secs 37.03 secs 3.87 0.02 False
Count - Duration of single invocation 7.18 secs 25.08 secs 3.49 0.01 False
Count - Duration of single invocation 7.10 secs 25.01 secs 3.52 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.368592218 > 12.229691438475.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2611.574027592 (T) = (0 -39381769731.29487) / Math.Sqrt((516670082564744.25 / (33)) + (2524701797321575.5 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.380472686120667 = (11649781964.808079 - 39381769731.29487) / 11649781964.808079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.983781547666666 > 7.417223067100001.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3983.170645280063 (T) = (0 -24984263458.24359) / Math.Sqrt((54098870033341.82 / (33)) + (483628430950479.6 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.536732877338905 = (7064221224.714645 - 24984263458.24359) / 7064221224.714645 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.030190412 > 9.9755976376.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -672.1034000786512 (T) = (0 -37158560480.14102) / Math.Sqrt((42134770156833024 / (33)) + (11232508349550664 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.9635181178861703 = (9375145861.565655 - 37158560480.14102) / 9375145861.565655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.08315895433333 > 7.541355660474999.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3622.8249250277245 (T) = (0 -25085121078.897438) / Math.Sqrt((52635269599694.54 / (33)) + (593306933144831.8 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.4916255428821024 = (7184367501.845961 - 25085121078.897438) / 7184367501.845961 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.009452057666667 > 7.454287711325.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3183.5885493092114 (T) = (0 -25009900690.79487) / Math.Sqrt((44794601561714.94 / (33)) + (787855657561277.8 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5240799625749815 = (7096859593.537879 - 25009900690.79487) / 7096859593.537879 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsDefined - Duration of single invocation 21.89 ns 45.76 ns 2.09 0.02 False
Parse - Duration of single invocation 867.08 ns 958.16 ns 1.11 0.01 False
TryParseGeneric - Duration of single invocation 760.12 ns 836.91 ns 1.10 0.01 False
GetName - Duration of single invocation 23.60 ns 47.35 ns 2.01 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Enum.IsDefined

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.75739592799014 > 22.96758031097912.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -523.6325674880006 (T) = (0 -45.80569504529459) / Math.Sqrt((0.06666122649874047 / (33)) + (0.0024841694364797534 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.1088231221086802 = (21.72097534642545 - 45.80569504529459) / 21.72097534642545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.Parse(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 958.1634391075158 > 911.4640054116962.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/14/2022 4:09:15 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -54.950901381236015 (T) = (0 -957.2331450063188) / Math.Sqrt((24.827724673418896 / (33)) + (45.933740343719855 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10024527995138882 = (870.0179518594357 - 957.2331450063188) / 870.0179518594357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 836.907182352521 > 792.0748773452038.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -52.53488153195229 (T) = (0 -835.457477368575) / Math.Sqrt((68.28133364634351 / (33)) + (4.171443784888442 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10362265433828652 = (757.0137076150373 - 835.457477368575) / 757.0137076150373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.GetName

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.3464701451827 > 25.286975731997035.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -387.0340576615878 (T) = (0 -47.290533885478766) / Math.Sqrt((0.0965986184308385 / (33)) + (0.018670984247205594 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9768062428590154 = (23.922695537972103 - 47.290533885478766) / 23.922695537972103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 195.63 μs 907.84 μs 4.64 0.01 False
Queue - Duration of single invocation 198.48 μs 911.31 μs 4.59 0.01 False
Array - Duration of single invocation 202.50 μs 875.08 μs 4.32 0.01 False
ImmutableArray - Duration of single invocation 203.46 μs 879.16 μs 4.32 0.01 False
List - Duration of single invocation 199.06 μs 875.58 μs 4.40 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 907.8396149509804 > 209.6678186738782.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -218.40068029586098 (T) = (0 -885045.2314638868) / Math.Sqrt((7472710.357723662 / (33)) + (250257968.6920166 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.435426039618168 = (199540.07203783237 - 885045.2314638868) / 199540.07203783237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 911.3096817401962 > 210.32190736997384.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -208.4820131765725 (T) = (0 -890633.5769901784) / Math.Sqrt((3569133.9576226645 / (33)) + (282651382.82557863 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.457047781399472 = (199825.89836865687 - 890633.5769901784) / 199825.89836865687 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 875.0763680555555 > 207.34344153906252.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -233.71263841849927 (T) = (0 -884014.9858527979) / Math.Sqrt((7121863.26514671 / (33)) + (216797719.48025993 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.409867503425369 = (200462.93571544686 - 884014.9858527979) / 200462.93571544686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 879.1562555555556 > 211.57266658327012.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -236.2201516545953 (T) = (0 -887838.9332464269) / Math.Sqrt((2487526.110021213 / (33)) + (217206177.28006008 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.39506259071661 = (202008.25697494007 - 887838.9332464269) / 202008.25697494007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 875.5783273809524 > 210.58968118215464.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -253.10054912460086 (T) = (0 -883027.2394083755) / Math.Sqrt((6988130.721255552 / (33)) + (184266311.1684676 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.432034311057983 = (199237.4556318779 - 883027.2394083755) / 199237.4556318779 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_String - Duration of single invocation 292.76 ns 337.97 ns 1.15 0.00 False
Format_OneArg - Duration of single invocation 1.11 μs 1.62 μs 1.46 0.01 False
Replace_String - Duration of single invocation 176.43 ns 245.93 ns 1.39 0.01 False
Replace_String - Duration of single invocation 42.42 ns 90.50 ns 2.13 0.01 False
Replace_Char - Duration of single invocation 391.52 ns 456.85 ns 1.17 0.00 False
Replace_Char - Duration of single invocation 11.32 ns 12.78 ns 1.13 0.21 False
IndexOfAny - Duration of single invocation 116.94 ns 464.38 ns 3.97 0.05 False
Format_MultipleArgs - Duration of single invocation 557.94 ns 944.90 ns 1.69 0.01 False
Format_OneArg - Duration of single invocation 2.57 μs 3.31 μs 1.29 0.02 False
Replace_String - Duration of single invocation 204.57 ns 403.98 ns 1.97 0.01 False
Replace_Char - Duration of single invocation 26.47 ns 75.88 ns 2.87 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 337.9710970354051 > 307.8126019794034.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -58.608506802045405 (T) = (0 -334.5822099466892) / Math.Sqrt((9.693922849979577 / (33)) + (4.726056675366902 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.13738982849133854 = (294.1666977895232 - 334.5822099466892) / 294.1666977895232 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:D5}, {0:E} - {0:F4}{0:G}{0:N}  {0:X} !!", o: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6222629860853433 > 1.1680643777912907.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -323.17498656973595 (T) = (0 -1630.895694009067) / Math.Sqrt((38.89163465000051 / (33)) + (35.09922606246636 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.4600594692503124 = (1117.00634690344 - 1630.895694009067) / 1117.00634690344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 245.934282535006 > 184.51800362068002.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -168.79563867616073 (T) = (0 -246.59880358095248) / Math.Sqrt((0.5382799252526629 / (33)) + (4.096964938195328 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.39944856294435765 = (176.21140934406554 - 246.59880358095248) / 176.21140934406554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.50221081590821 > 44.652851134772966.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -397.63105977460975 (T) = (0 -90.68671534354563) / Math.Sqrt((0.4283105149410602 / (33)) + (0.04477534856110255 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.1350868295117977 = (42.47448585699054 - 90.68671534354563) / 42.47448585699054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 456.8499922062325 > 411.56293123522164.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -111.48767186018905 (T) = (0 -456.72954849510336) / Math.Sqrt((5.629895727176973 / (33)) + (4.161257834629104 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.1632817975800377 = (392.62158957978437 - 456.72954849510336) / 392.62158957978437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.780656663594304 > 11.595723346102481.
IsChangePoint: Marked as a change because one of 7/2/2022 7:23:59 AM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -29.95504205088192 (T) = (0 -12.968822138390218) / Math.Sqrt((0.028852536282161376 / (33)) + (0.08612169249713275 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.17571411119279948 = (11.030591548512533 - 12.968822138390218) / 11.030591548512533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.IndexOfAny

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 464.38011231035165 > 120.64901186331139.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -421.13922751046016 (T) = (0 -467.5382085764859) / Math.Sqrt((13.807514609548539 / (33)) + (7.57721334548826 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.1477924198776623 = (112.71977024112401 - 467.5382085764859) / 112.71977024112401 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_MultipleArgs

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 944.9017408485437 > 592.0165509863924.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -212.4701629670811 (T) = (0 -958.1725672125147) / Math.Sqrt((20.308472580178748 / (33)) + (73.79074035178907 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.7009206663235956 = (563.325842400121 - 958.1725672125147) / 563.325842400121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3067170998872917 > 2.692747392440076.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/18/2022 2:17:16 AM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -34.755299875880795 (T) = (0 -3280.9201971652574) / Math.Sqrt((208.6004176873463 / (33)) + (10934.1369826367 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2801837418786179 = (2562.8510110202146 - 3280.9201971652574) / 2562.8510110202146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 403.98403556412046 > 213.47585164080394.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -848.2212012953711 (T) = (0 -404.6842553634864) / Math.Sqrt((0.8917318487794103 / (33)) + (0.7610489161139472 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9892701027701697 = (203.4335381605248 - 404.6842553634864) / 203.4335381605248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.87784605701911 > 28.08915217993635.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -683.7280781656918 (T) = (0 -75.92039007640683) / Math.Sqrt((0.06297942315294768 / (33)) + (0.08509964788285912 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8430669447604666 = (26.70369412733025 - 75.92039007640683) / 26.70369412733025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 111.78 μs 449.72 μs 4.02 0.01 False
Queue - Duration of single invocation 111.50 μs 443.61 μs 3.98 0.03 False
Array - Duration of single invocation 110.61 μs 446.24 μs 4.03 0.01 False
ICollection - Duration of single invocation 109.05 μs 443.90 μs 4.07 0.01 False
List - Duration of single invocation 104.21 μs 462.91 μs 4.44 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 449.7150969047619 > 115.67781973091958.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -251.00606145774094 (T) = (0 -451387.13950224966) / Math.Sqrt((2121921.3786833943 / (33)) + (46414934.02997113 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.102437582940242 = (110029.00845568448 - 451387.13950224966) / 110029.00845568448 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 443.60516940476197 > 117.04209660239363.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -210.8701170984569 (T) = (0 -448459.6956179981) / Math.Sqrt((6577714.670580802 / (33)) + (62533607.21518635 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.1466407982132627 = (108150.11896165057 - 448459.6956179981) / 108150.11896165057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 446.2362605952381 > 112.6620831685094.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -277.31499040059674 (T) = (0 -448153.4498415005) / Math.Sqrt((2274540.721330532 / (33)) + (37359187.59952687 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.1551330900758106 = (107855.3779449033 - 448153.4498415005) / 107855.3779449033 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 443.89582933673455 > 114.11851264592315.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -188.46579152178373 (T) = (0 -450440.03107039665) / Math.Sqrt((3645471.096927401 / (33)) + (83056390.18045662 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.177762711532189 = (107818.48136731498 - 450440.03107039665) / 107818.48136731498 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 462.9075921743697 > 112.31188388068483.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -216.8534850244752 (T) = (0 -449061.35543727846) / Math.Sqrt((6130965.375202742 / (33)) + (59885150.08875136 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.199315523131936 = (106936.79790518795 - 449061.35543727846) / 106936.79790518795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SplitWords - Duration of single invocation 105.34 μs 157.70 μs 1.50 0.04 False
MatchesWords - Duration of single invocation 15.34 μs 53.90 μs 3.51 0.01 False
MatchWord - Duration of single invocation 518.24 ns 1.73 μs 3.33 0.01 False
ReplaceWords - Duration of single invocation 17.26 μs 56.01 μs 3.25 0.01 False
MatchesWord - Duration of single invocation 8.13 μs 27.80 μs 3.42 0.01 False
MatchesWord - Duration of single invocation 7.65 μs 21.64 μs 2.83 0.01 False
Backtracking - Duration of single invocation 387.01 ns 618.15 ns 1.60 0.01 False
MatchesWords - Duration of single invocation 104.69 μs 155.60 μs 1.49 0.02 False
MatchesWords - Duration of single invocation 10.77 μs 31.70 μs 2.94 0.01 False
SplitWords - Duration of single invocation 17.08 μs 54.78 μs 3.21 0.01 False
OneNodeBacktracking - Duration of single invocation 236.55 ns 422.59 ns 1.79 0.01 False
MatchWord - Duration of single invocation 2.87 μs 4.16 μs 1.45 0.01 False
Backtracking - Duration of single invocation 362.96 ns 467.69 ns 1.29 0.01 False
OneNodeBacktracking - Duration of single invocation 274.24 ns 661.85 ns 2.41 0.02 False
MatchesWord - Duration of single invocation 7.33 μs 21.24 μs 2.90 0.01 False
SplitWords - Duration of single invocation 12.56 μs 33.52 μs 2.67 0.02 False
ReplaceWords - Duration of single invocation 13.11 μs 34.25 μs 2.61 0.01 False
MatchWord - Duration of single invocation 385.21 ns 1.09 μs 2.82 0.01 False
OneNodeBacktracking - Duration of single invocation 1.61 μs 1.78 μs 1.10 0.03 False
ReplaceWords - Duration of single invocation 107.44 μs 156.72 μs 1.46 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.70153379166666 > 113.01027965657478.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -104.65361632683974 (T) = (0 -156318.67151396308) / Math.Sqrt((3538773.4336867626 / (33)) + (3172433.0180678344 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.47178563423556413 = (106210.21694858027 - 156318.67151396308) / 106210.21694858027 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.90483379356897 > 16.132622830017837.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -500.65796645885337 (T) = (0 -53320.26412462531) / Math.Sqrt((25417.956757494747 / (33)) + (130026.26432616022 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.488198460154431 = (15285.903234492194 - 53320.26412462531) / 15285.903234492194 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.725704478688227 > 544.7127237730542.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -272.1761602876511 (T) = (0 -1754.8604131291552) / Math.Sqrt((23.39318077834236 / (33)) + (514.8103942113468 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.360183150594341 = (522.2514174022805 - 1754.8604131291552) / 522.2514174022805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.00835578640111 > 18.267343186682485.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -175.54434459031788 (T) = (0 -55436.346312861126) / Math.Sqrt((1358540.6947370577 / (33)) + (134142.3196712412 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.1404060203068025 = (17652.60477606818 - 55436.346312861126) / 17652.60477606818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.8004941974655 > 8.527990609802162.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1025.4546978010649 (T) = (0 -27833.835742921434) / Math.Sqrt((725.8164298619754 / (33)) + (9032.638214596283 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.4258190151571606 = (8124.724516903456 - 27833.835742921434) / 8124.724516903456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.638970862268522 > 7.992702327968689.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1734.6486465178355 (T) = (0 -21654.479460845676) / Math.Sqrt((742.6344595971557 / (33)) + (1116.083775494538 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.840649819962806 = (7623.072477525304 - 21654.479460845676) / 7623.072477525304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 618.1539188593757 > 404.93127560226725.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -272.3942206235648 (T) = (0 -625.3902652179845) / Math.Sqrt((4.1254871473439 / (33)) + (16.835303671965825 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.6202932877862886 = (385.9734962380906 - 625.3902652179845) / 385.9734962380906 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.6046066017316 > 110.13694037318932.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -100.37784166382204 (T) = (0 -155020.78336423263) / Math.Sqrt((2970119.156484972 / (33)) + (4066595.501750714 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.47367393681355263 = (105193.4077760959 - 155020.78336423263) / 105193.4077760959 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.696157919202165 > 11.285577208878363.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1098.7355749205967 (T) = (0 -31748.092051090403) / Math.Sqrt((6336.978362959209 / (33)) + (4437.441016712664 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.933389285445508 = (10823.006754887178 - 31748.092051090403) / 10823.006754887178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.77896585025552 > 17.809603484963908.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -386.39148594753624 (T) = (0 -55316.01459104746) / Math.Sqrt((2111.283191609021 / (33)) + (254814.8368418619 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.2655153808964066 = (16939.44389747839 - 55316.01459104746) / 16939.44389747839 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 422.58946921213436 > 249.24654427792723.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -545.7777787831001 (T) = (0 -423.6528737397293) / Math.Sqrt((1.355275855512515 / (33)) + (1.9689623443891264 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.7865913138279274 = (237.12914669444805 - 423.6528737397293) / 237.12914669444805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.164280308454576 > 3.0041208794315537.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -106.14377809392067 (T) = (0 -4218.972939892944) / Math.Sqrt((2344.430790220888 / (33)) + (2144.632138464542 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.4529253204420121 = (2903.7782469159797 - 4218.972939892944) / 2903.7782469159797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 467.68617285109906 > 383.0109815518568.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -114.66822452851733 (T) = (0 -467.9443847911006) / Math.Sqrt((15.08434219197225 / (33)) + (8.47680685982627 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2769002855015503 = (366.46901101388505 - 467.9443847911006) / 366.46901101388505 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 661.8515208226697 > 285.6077955535156.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -643.5274453306642 (T) = (0 -660.0642734629248) / Math.Sqrt((1.7807871591559863 / (33)) + (8.053505000176104 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.4270504360649574 = (271.9614984734742 - 660.0642734629248) / 271.9614984734742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.242097148653585 > 7.64093211720581.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -898.2434773849042 (T) = (0 -21273.078093572112) / Math.Sqrt((611.292765824904 / (33)) + (5828.247391925571 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9221832108282941 = (7279.857749761778 - 21273.078093572112) / 7279.857749761778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.51729928141987 > 13.219960202925854.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -480.6462100278004 (T) = (0 -33611.815813187015) / Math.Sqrt((19706.274126764187 / (33)) + (33817.178432623965 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.6522481332929178 = (12672.952953107002 - 33611.815813187015) / 12672.952953107002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.24588680101671 > 13.623784281438006.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -577.6637036611528 (T) = (0 -34086.375627981346) / Math.Sqrt((10601.56560803813 / (33)) + (26509.259367703544 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.6354628039082306 = (12933.734286605499 - 34086.375627981346) / 12933.734286605499 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0854982866711405 > 406.6502928144849.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -680.878137386783 (T) = (0 -1085.5161592626216) / Math.Sqrt((7.578166246426546 / (33)) + (21.455086705929507 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8106270561174949 = (386.2184977191947 - 1085.5161592626216) / 386.2184977191947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7766433255830465 > 1.6970792043423504.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -20.92361942495888 (T) = (0 -1748.0496313974995) / Math.Sqrt((121.67166025349879 / (33)) + (888.1869784492724 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.07949216903881359 = (1619.3259029882297 - 1748.0496313974995) / 1619.3259029882297 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.72169866071428 > 112.55138624731741.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -109.32436622437952 (T) = (0 -156542.13512925798) / Math.Sqrt((1154469.952925853 / (33)) + (4508033.4120645225 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.4679736544158848 = (106638.24562406537 - 156542.13512925798) / 106638.24562406537 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 214.53 ns 695.85 ns 3.24 0.12 False
LastIndexOfValue - Duration of single invocation 179.06 ns 600.95 ns 3.36 0.37 False
IndexOfAnyTwoValues - Duration of single invocation 318.57 ns 1.38 μs 4.33 0.03 False
IndexOfAnyThreeValues - Duration of single invocation 420.26 ns 1.67 μs 3.98 0.17 False
LastIndexOfAnyValues - Duration of single invocation 316.87 ns 1.03 μs 3.26 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Byte>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 695.8523947577379 > 224.9233791001632.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -757.6141445243205 (T) = (0 -697.4919904473577) / Math.Sqrt((2.7582436860121526 / (33)) + (8.392123551304367 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.2510718402487764 = (214.54216477541286 - 697.4919904473577) / 214.54216477541286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 600.9515324770258 > 188.87598503171583.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -655.8990556615088 (T) = (0 -604.3536115100826) / Math.Sqrt((4.956905440649008 / (33)) + (6.941764976698567 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.344483924616306 = (180.70160453213023 - 604.3536115100826) / 180.70160453213023 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3783111912645853 > 335.1931279452337.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -691.3650126004583 (T) = (0 -1371.9677670930964) / Math.Sqrt((52.3782675506699 / (33)) + (18.81794040904915 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.274583254353899 = (320.9594211776485 - 1371.9677670930964) / 320.9594211776485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6706800499108734 > 458.2756817948821.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -762.3724820708376 (T) = (0 -1670.5444472933289) / Math.Sqrt((26.1492253061541 / (33)) + (48.00884153188624 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.8661497294767284 = (432.0951241377379 - 1670.5444472933289) / 432.0951241377379 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0317213571271204 > 345.1459806334516.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -909.0770969979969 (T) = (0 -1030.9653819077469) / Math.Sqrt((19.003345687356436 / (33)) + (0.6112463259778569 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.151180425394465 = (327.1679950787618 - 1030.9653819077469) / 327.1679950787618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 855.32 μs 920.04 μs 1.08 0.01 False
SerializeToWriter - Duration of single invocation 821.83 μs 891.15 μs 1.08 0.02 False
SerializeToStream - Duration of single invocation 829.92 μs 897.99 μs 1.08 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 920.0365275735294 > 896.607983832237.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -60.85299477738723 (T) = (0 -919852.9679729411) / Math.Sqrt((18385476.987400625 / (33)) + (15464792.746051908 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.0764299120216229 = (854540.5118345164 - 919852.9679729411) / 854540.5118345164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;.SerializeToWriter

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 891.148626446759 > 864.3273706809844.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -66.72152214472825 (T) = (0 -893045.3025069415) / Math.Sqrt((14451461.574626397 / (33)) + (14391902.468100278 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.0803716229019896 = (826609.3662365268 - 893045.3025069415) / 826609.3662365268 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;.SerializeToStream

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 897.9938303571429 > 878.0185003125001.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -59.079868783104104 (T) = (0 -901836.3176047949) / Math.Sqrt((22791511.47262006 / (33)) + (15065493.85914724 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.07971476352939082 = (835254.2246035946 - 901836.3176047949) / 835254.2246035946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteFormat - Duration of single invocation 253.73 ns 386.20 ns 1.52 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_StreamWriter.WriteFormat

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 386.20144496583396 > 267.1578042101552.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -241.34324729679122 (T) = (0 -383.9741433409648) / Math.Sqrt((4.1275720983005675 / (33)) + (4.164227916971737 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.5053220180241552 = (255.07774332893825 - 383.9741433409648) / 255.07774332893825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in BenchmarksGame.ReverseComplement_1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 992.51 μs 1.45 ms 1.46 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.ReverseComplement_1*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

BenchmarksGame.ReverseComplement_1.RunBench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4505416850378787 > 1.0509574483789061.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -61.66399857304926 (T) = (0 -1441727.074768014) / Math.Sqrt((41777359.04624342 / (33)) + (1310991718.4650965 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.4440455020722347 = (998394.491516442 - 1441727.074768014) / 998394.491516442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HtmlDecode_Entities - Duration of single invocation 780.59 ns 1.12 μs 1.43 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1167312109606617 > 818.308127005561.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -188.8671067979763 (T) = (0 -1119.1811176489018) / Math.Sqrt((65.12349642973797 / (33)) + (31.479816946898595 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.4308844012054323 = (782.1604014314926 - 1119.1811176489018) / 782.1604014314926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 92.20 ns 342.20 ns 3.71 0.03 False
ToArray - Duration of single invocation 1.33 μs 1.42 μs 1.07 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 342.20043600284316 > 95.94809395118484.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -157.84896643562612 (T) = (0 -345.9047713693057) / Math.Sqrt((1.2653093068616628 / (33)) + (67.0165863548718 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.8177771231712927 = (90.60371001489338 - 345.9047713693057) / 90.60371001489338 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.424362532308581 > 1.384643221350327.
IsChangePoint: Marked as a change because one of 8/2/2022 9:59:24 PM, 8/5/2022 10:47:59 PM, 9/24/2022 1:05:07 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -16.550406253180444 (T) = (0 -1415.832969489466) / Math.Sqrt((440.1469401403985 / (46)) + (220.3532089620296 / (13))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (13) - 2, .025) and -0.06405240344086409 = (1330.6045500306532 - 1415.832969489466) / 1330.6045500306532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse2 - Duration of single invocation 74.37 ns 79.01 ns 1.06 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Version.TryParse2

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.00876674728127 > 77.59512286537931.
IsChangePoint: Marked as a change because one of 7/2/2022 10:25:38 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -31.79831584104334 (T) = (0 -78.7210485223861) / Math.Sqrt((0.23937704541623714 / (33)) + (0.30787853717171965 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.05911821222873887 = (74.32697088338297 - 78.7210485223861) / 74.32697088338297 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfChar - Duration of single invocation 54.12 ns 244.84 ns 4.52 0.01 False
IndexOfShort - Duration of single invocation 55.70 ns 244.50 ns 4.39 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Array.IndexOfChar

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 244.83882077342804 > 56.85834557275816.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1250.7414984599989 (T) = (0 -244.9644342023062) / Math.Sqrt((0.39571036553746225 / (33)) + (0.2903897176906461 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.4847846380368 = (54.621225760695296 - 244.9644342023062) / 54.621225760695296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfShort

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 244.50429188126404 > 58.37503674777662.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1213.7487749851643 (T) = (0 -244.5604865979364) / Math.Sqrt((0.7199114682162862 / (33)) + (0.06917098840406659 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.473220246194578 = (54.67213173909488 - 244.5604865979364) / 54.67213173909488 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Diagnostics.Perf_Process

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCurrentProcessName - Duration of single invocation 115.53 μs 131.63 μs 1.14 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Diagnostics.Perf_Process*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Diagnostics.Perf_Process.GetCurrentProcessName

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.62573413738855 > 121.91354721068804.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -26.87202426316709 (T) = (0 -128095.38503667821) / Math.Sqrt((2616621.580494279 / (33)) + (2600038.412858158 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.09748731884788957 = (116716.96140521154 - 128095.38503667821) / 116716.96140521154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 224.57 ns 863.89 ns 3.85 0.01 False
LastIndexOf_Word_NotFound - Duration of single invocation 234.76 ns 680.55 ns 2.90 0.23 False
IndexOf_Word_NotFound - Duration of single invocation 384.92 ns 1.42 μs 3.70 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 863.8925674565793 > 235.23098774271872.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3074.9790968816897 (T) = (0 -864.4752093874371) / Math.Sqrt((0.714436660945163 / (33)) + (0.5638396008614852 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.8532516541596284 = (224.34953306364676 - 864.4752093874371) / 224.34953306364676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 680.5457049455837 > 245.7165701963424.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -80.83840042590685 (T) = (0 -691.4986385995829) / Math.Sqrt((117.69111480774848 / (33)) + (718.6913501023138 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.882453434562824 = (239.89932684010944 - 691.4986385995829) / 239.89932684010944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.42335479954801 > 403.3260422410841.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -921.4486392262547 (T) = (0 -1425.0823498990965) / Math.Sqrt((40.62819646910489 / (33)) + (1.404257819801703 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.745485150839278 = (380.4800426400751 - 1425.0823498990965) / 380.4800426400751 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddChainedConfigurationEmpty - Duration of single invocation 49.20 ms 91.00 ms 1.85 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.99967489285714 > 52.231357535.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -130.64670084601232 (T) = (0 -89673998.27751653) / Math.Sqrt((231282551760.28595 / (33)) + (2275045773276.9707 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.8112338441527194 = (49509895.4599456 - 89673998.27751653) / 49509895.4599456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileName - Duration of single invocation 60.63 ns 86.94 ns 1.43 0.00 False
GetFileNameWithoutExtension - Duration of single invocation 71.41 ns 105.91 ns 1.48 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.93543728877641 > 63.76860622046939.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -282.85619496482656 (T) = (0 -86.77502633241353) / Math.Sqrt((0.08375002256146288 / (33)) + (0.1537327539406312 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.42786091521364583 = (60.772744325332056 - 86.77502633241353) / 60.772744325332056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.90596422877299 > 74.96728344661649.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -86.91895729533921 (T) = (0 -106.74460779134743) / Math.Sqrt((0.24707406290510625 / (33)) + (4.113797588666301 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.4958171244478712 = (71.36207096890169 - 106.74460779134743) / 71.36207096890169 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 196.68 ns 709.93 ns 3.61 0.20 False
IndexOfAnyThreeValues - Duration of single invocation 467.32 ns 2.21 μs 4.72 0.01 False
IndexOfAnyFourValues - Duration of single invocation 622.04 ns 2.84 μs 4.56 0.05 False
IndexOfValue - Duration of single invocation 188.55 ns 990.70 ns 5.25 0.01 False
IndexOfAnyTwoValues - Duration of single invocation 377.99 ns 1.60 μs 4.23 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 709.932751426317 > 232.32650481075953.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -85.40948070967276 (T) = (0 -724.0214127568753) / Math.Sqrt((137.42638711517878 / (33)) + (822.6346933594562 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.3998032289961593 = (212.9597991383322 - 724.0214127568753) / 212.9597991383322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2060743097134536 > 490.32338291358496.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -5599.715785489925 (T) = (0 -2205.8102620463123) / Math.Sqrt((0.20980864077718503 / (33)) + (2.3416885961063723 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.7235091512361964 = (466.9854956180251 - 2205.8102620463123) / 466.9854956180251 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8379928037185076 > 650.1998501316313.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -736.1336488963775 (T) = (0 -2844.8948158767134) / Math.Sqrt((276.85339335761057 / (33)) + (24.025237460135994 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.754528480114525 = (598.3547743536046 - 2844.8948158767134) / 598.3547743536046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 990.6965235887994 > 195.7561247621765.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -4187.873111659416 (T) = (0 -988.1705165626058) / Math.Sqrt((0.4965300006794466 / (33)) + (0.5612161544785157 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.29460023049704 = (186.6374180378562 - 988.1705165626058) / 186.6374180378562 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5971916368286445 > 397.30636022301485.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -785.5066641116184 (T) = (0 -1597.8199323920953) / Math.Sqrt((71.31441692486716 / (33)) + (6.383769661777367 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.2130701710991483 = (379.25310225137787 - 1597.8199323920953) / 379.25310225137787 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 4.46 ms 22.74 ms 5.10 0.01 False
Count - Duration of single invocation 6.72 ms 24.98 ms 3.72 0.01 False
Count - Duration of single invocation 8.58 ms 26.92 ms 3.14 0.01 False
Count - Duration of single invocation 4.45 ms 22.73 ms 5.11 0.01 False
Count - Duration of single invocation 4.36 ms 22.64 ms 5.19 0.01 False
Count - Duration of single invocation 10.58 ms 28.87 ms 2.73 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w.+-]+@[\w.-]+.[\w.-]+", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.737569748251744 > 4.676053644283354.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3706.5301712210844 (T) = (0 -22743861.382664777) / Math.Sqrt((307899635.4797428 / (33)) + (389993716.43740827 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.0981308672998775 = (4461215.683682637 - 22743861.382664777) / 4461215.683682637 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.980049580000006 > 7.067008177343749.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1372.0160790213226 (T) = (0 -25010556.75901521) / Math.Sqrt((369729776.04239416 / (33)) + (4319067561.463189 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.7105016905325936 = (6740478.470291513 - 25010556.75901521) / 6740478.470291513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.9202771025641 > 8.99128662267857.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2354.267153298113 (T) = (0 -26874664.000597198) / Math.Sqrt((784801574.866879 / (33)) + (951763688.4027327 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.132356104985289 = (8579696.273302047 - 26874664.000597198) / 8579696.273302047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.7257067939394 > 4.677845373290598.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3110.6108581727785 (T) = (0 -22731140.074690692) / Math.Sqrt((735666322.3491813 / (33)) + (317119960.5760814 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.093095493446654 = (4463128.583380994 - 22731140.074690692) / 4463128.583380994 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.643372733333333 > 4.590216577050782.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -4064.6321752615354 (T) = (0 -22643718.088138144) / Math.Sqrt((358776660.7675369 / (33)) + (242632059.79459637 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.177304791323263 = (4373649.804447896 - 22643718.088138144) / 4373649.804447896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\s?#]+[^\s?#]+(?:\?[^\s#]*)?(?:#[^\s]*)?", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.873152427350423 > 11.195190190434785.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -435.0465103558614 (T) = (0 -28870717.63394765) / Math.Sqrt((53761716219.40016 / (33)) + (3318825436.9150558 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.714397107684406 = (10636143.677067444 - 28870717.63394765) / 10636143.677067444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExpandEnvironmentVariables - Duration of single invocation 344.92 ns 512.32 ns 1.49 0.01 False
SetEnvironmentVariable - Duration of single invocation 164.44 ns 232.88 ns 1.42 0.01 False
GetEnvironmentVariable - Duration of single invocation 111.75 ns 181.31 ns 1.62 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Environment.ExpandEnvironmentVariables

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 512.3200878978137 > 358.9525885424446.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -121.64186956108061 (T) = (0 -513.5820567590624) / Math.Sqrt((3.8236260264034705 / (33)) + (48.33898649263747 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.49893868122954677 = (342.63046460164884 - 513.5820567590624) / 342.63046460164884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.SetEnvironmentVariable

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 232.88421692411208 > 170.7486198270786.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -179.95089133442775 (T) = (0 -233.32355694501035) / Math.Sqrt((2.2528314791781323 / (33)) + (2.2276260560266805 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.43391173625589075 = (162.7182141309793 - 233.32355694501035) / 162.7182141309793 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.GetEnvironmentVariable

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 181.3100651862157 > 118.02507589653032.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -218.18603517753803 (T) = (0 -182.28489126731336) / Math.Sqrt((0.4729983409236218 / (33)) + (2.267856131192206 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.6166769196136012 = (112.75282590839542 - 182.28489126731336) / 112.75282590839542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in BenchmarksGame.RegexRedux_5

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 54.95 ms 84.91 ms 1.55 0.14 False
RunBench - Duration of single invocation 14.79 ms 31.10 ms 2.10 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.RegexRedux_5*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

BenchmarksGame.RegexRedux_5.RunBench(options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.9130309736842 > 58.23668608406251.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -82.28671378264306 (T) = (0 -85259288.47708577) / Math.Sqrt((2541591547459.0073 / (33)) + (1464595322150.674 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.54425247468308 = (55210718.37335611 - 85259288.47708577) / 55210718.37335611 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### BenchmarksGame.RegexRedux_5.RunBench(options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.098180675000002 > 15.637730585955884.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -145.50868756370616 (T) = (0 -30498452.875557046) / Math.Sqrt((13030210649.248922 / (33)) + (288203980020.17206 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.0457481292303965 = (14908214.965363532 - 30498452.875557046) / 14908214.965363532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 400.51 ns 555.14 ns 1.39 0.01 False
IsMatch - Duration of single invocation 315.36 ns 375.64 ns 1.19 0.02 False
IsMatch - Duration of single invocation 169.22 ns 333.24 ns 1.97 0.02 False
IsMatch - Duration of single invocation 485.10 ns 558.50 ns 1.15 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 555.139798568095 > 422.878115255711.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -146.55667747243498 (T) = (0 -560.4438153783116) / Math.Sqrt((19.813721972771635 / (33)) + (13.981922126441843 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.3869318971272988 = (404.0889221303067 - 560.4438153783116) / 404.0889221303067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 375.64296056529236 > 331.5580602395882.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -43.94212530223465 (T) = (0 -376.8131874164996) / Math.Sqrt((57.74760370142993 / (33)) + (3.572906624582611 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.19076928456509143 = (316.4451689347398 - 376.8131874164996) / 316.4451689347398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 333.237188243872 > 179.880392955831.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -303.9489592159191 (T) = (0 -334.1177441050816) / Math.Sqrt((3.896453740811115 / (33)) + (4.357358474110609 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9463450811207478 = (171.66418604078643 - 334.1177441050816) / 171.66418604078643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 558.4955676435555 > 510.6122071258136.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -53.481610492387915 (T) = (0 -558.4389181109071) / Math.Sqrt((9.705316043419993 / (33)) + (38.452641991563375 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.14616223378589338 = (487.22502072531665 - 558.4389181109071) / 487.22502072531665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 542.57 ns 2.17 μs 4.00 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.171631883060516 > 568.8153524743115.
IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -265.9575753656801 (T) = (0 -2180.922785635938) / Math.Sqrt((46.66402868730114 / (33)) + (956.3031512391971 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.059418413496486 = (537.2500598570845 - 2180.922785635938) / 537.2500598570845 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 40.11 μs 115.54 μs 2.88 0.02 False
ReadLineAsync - Duration of single invocation 709.97 μs 1.66 ms 2.34 0.01 False
ReadLineAsync - Duration of single invocation 247.90 μs 403.78 μs 1.63 0.01 False
ReadLine - Duration of single invocation 146.81 μs 307.17 μs 2.09 0.01 False
ReadLine - Duration of single invocation 58.53 μs 155.05 μs 2.65 0.01 False
ReadLine - Duration of single invocation 45.63 μs 124.19 μs 2.72 0.01 False
ReadLineAsync - Duration of single invocation 51.13 μs 130.70 μs 2.56 0.01 False
ReadLine - Duration of single invocation 29.74 μs 107.46 μs 3.61 0.06 False
ReadLine - Duration of single invocation 283.59 μs 1.27 ms 4.47 0.01 False
ReadLineAsync - Duration of single invocation 39.89 μs 114.15 μs 2.86 0.03 False
ReadLine - Duration of single invocation 37.53 μs 114.30 μs 3.05 0.01 False
ReadLine - Duration of single invocation 250.74 μs 736.72 μs 2.94 0.01 False
ReadLine - Duration of single invocation 38.83 μs 112.61 μs 2.90 0.03 False
ReadLineAsync - Duration of single invocation 31.89 μs 107.87 μs 3.38 0.05 False
ReadLineAsync - Duration of single invocation 492.17 μs 942.40 μs 1.91 0.01 False
ReadLineAsync - Duration of single invocation 81.71 μs 178.42 μs 2.18 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.53961019564075 > 42.07968713469471.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -416.73236712997283 (T) = (0 -115494.13274886998) / Math.Sqrt((1000975.3811590442 / (33)) + (73375.10533883385 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9155102326120792 = (39613.694871307605 - 115494.13274886998) / 39613.694871307605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6623957577083335 > 769.0080947301137.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -275.50319822247974 (T) = (0 -1671958.6897263753) / Math.Sqrt((263442563.599615 / (33)) + (100061422.58099732 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.3083569972446267 = (724306.8085751515 - 1671958.6897263753) / 724306.8085751515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 403.78283301282056 > 257.4976296471229.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -141.63794201089195 (T) = (0 -402694.876828514) / Math.Sqrt((11064866.083671102 / (33)) + (23328709.15790805 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.6406551429433489 = (245447.60583023928 - 402694.876828514) / 245447.60583023928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 307.17057788461534 > 155.3173208349434.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -248.3104466043821 (T) = (0 -307187.4890160637) / Math.Sqrt((1807466.422491923 / (33)) + (9230615.787050743 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.0723579450175438 = (148230.90275240224 - 307187.4890160637) / 148230.90275240224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.04655672442246 > 61.91296828691134.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -534.2904545769914 (T) = (0 -155734.40501291188) / Math.Sqrt((323451.23109348747 / (33)) + (594197.1204264555 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.6313606394491158 = (59183.98363118915 - 155734.40501291188) / 59183.98363118915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 124.18779021400228 > 47.61622361749031.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -580.0279456979456 (T) = (0 -124051.426494083) / Math.Sqrt((550540.5898065688 / (33)) + (50855.24166014731 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.765102343653628 = (44863.23147452454 - 124051.426494083) / 44863.23147452454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.70439887548892 > 54.17575101526582.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -523.3431376026322 (T) = (0 -130348.35409581201) / Math.Sqrt((630855.8638463803 / (33)) + (92501.53174321236 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.5289296713119758 = (51542.89404504831 - 130348.35409581201) / 51542.89404504831 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.45962891389432 > 30.756109910803815.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -332.6628461822774 (T) = (0 -107334.41080921995) / Math.Sqrt((1723439.3408810538 / (33)) + (40070.84785930827 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5543809542059366 = (30197.77907660967 - 107334.41080921995) / 30197.77907660967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2675526230769234 > 296.85960638310183.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1252.0624430946716 (T) = (0 -1267438.4408109952) / Math.Sqrt((15685572.12582968 / (33)) + (3685227.3370776502 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.464323026502146 = (283903.8378914192 - 1267438.4408109952) / 283903.8378914192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.15107790450124 > 41.90768873023666.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -410.12775801661576 (T) = (0 -113931.076779364) / Math.Sqrt((980673.3587178058 / (33)) + (81537.72480544362 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.877534573540171 = (39593.295533960154 - 113931.076779364) / 39593.295533960154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.29783778892946 > 40.89042833803338.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -455.09833234634425 (T) = (0 -114971.27493792158) / Math.Sqrt((738295.2965982375 / (33)) + (142621.6256609058 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9470699800658735 = (39012.06137471894 - 114971.27493792158) / 39012.06137471894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 736.7202845238095 > 265.5533818545082.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -581.9919673880908 (T) = (0 -736369.0582402479) / Math.Sqrt((21056303.504567586 / (33)) + (1200883.7887687597 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.888332579443036 = (254946.07632139223 - 736369.0582402479) / 254946.07632139223 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 112.61295720663264 > 40.852729418960166.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -459.5591628951318 (T) = (0 -112717.29411936208) / Math.Sqrt((773673.384541656 / (33)) + (69714.60231494962 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9325741063965618 = (38436.29863385273 - 112717.29411936208) / 38436.29863385273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.87171942528738 > 33.16816147600842.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -384.9451354620132 (T) = (0 -107696.36815731201) / Math.Sqrt((1257111.8996764608 / (33)) + (49708.96513019528 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5078661103981137 = (30701.390750939856 - 107696.36815731201) / 30701.390750939856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 942.4019848039217 > 517.1195426836567.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -258.1938304932049 (T) = (0 -945706.8693055108) / Math.Sqrt((45018701.46299593 / (33)) + (45100241.36184308 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.9253159297939948 = (491195.6810156864 - 945706.8693055108) / 491195.6810156864 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 178.41685546875 > 87.04581651519624.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -360.56804559447374 (T) = (0 -177866.6585747604) / Math.Sqrt((1422219.8412736498 / (33)) + (710492.3917620127 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.1643360727242844 = (82180.70234853905 - 177866.6585747604) / 82180.70234853905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 146.20 ns 686.53 ns 4.70 0.01 False
IndexOfString - Duration of single invocation 128.98 ns 457.39 ns 3.55 0.05 False
IndexOfString - Duration of single invocation 47.56 ns 208.39 ns 4.38 0.01 False
IndexOfString - Duration of single invocation 46.95 ns 167.05 ns 3.56 0.02 False
IndexOfString - Duration of single invocation 356.19 ns 1.83 μs 5.13 0.01 False
IndexOfString - Duration of single invocation 360.62 ns 1.83 μs 5.06 0.01 False
IndexOfString - Duration of single invocation 24.85 ns 76.22 ns 3.07 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 686.5331568507805 > 153.97079885611114.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1052.31039111784 (T) = (0 -687.8876296271526) / Math.Sqrt((1.4965601260160764 / (33)) + (5.6948367259446 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.6855886375027147 = (146.80922352453396 - 687.8876296271526) / 146.80922352453396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 457.38993358474676 > 135.3348798311057.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -539.557182896935 (T) = (0 -456.51369248784636) / Math.Sqrt((11.720113786679262 / (33)) + (0.47923390573678637 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.602345019264685 = (126.72680991034846 - 456.51369248784636) / 126.72680991034846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 208.38526495603978 > 50.231157563696904.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -993.9889174653148 (T) = (0 -208.57159829915716) / Math.Sqrt((0.1578613349837372 / (33)) + (0.5559401231554809 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.3648015433617964 = (47.784898403081584 - 208.57159829915716) / 47.784898403081584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.04543524346246 > 49.3651638614837.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -798.2566257830242 (T) = (0 -167.31760520460014) / Math.Sqrt((0.6086601595580897 / (33)) + (0.10830047476966043 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.538289985741641 = (47.28770278265636 - 167.31760520460014) / 47.28770278265636 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8260117209965852 > 377.4954891267641.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2962.5371296257426 (T) = (0 -1826.9419081835536) / Math.Sqrt((5.377113026921399 / (33)) + (2.1441650928784295 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.08426897984158 = (359.3322688919733 - 1826.9419081835536) / 359.3322688919733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8261341795470885 > 378.41562349657676.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3038.038251098393 (T) = (0 -1826.9532526124385) / Math.Sqrt((4.754258178995588 / (33)) + (2.3244757940946443 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.088901082293897 = (359.0074208691265 - 1826.9532526124385) / 359.0074208691265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.21853823729465 > 27.295727927989013.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -406.61231167008776 (T) = (0 -76.21054799403531) / Math.Sqrt((0.4973131742904714 / (33)) + (0.011480599392439695 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9806199632759405 = (25.56869004872188 - 76.21054799403531) / 25.56869004872188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 202.19 ns 871.55 ns 4.31 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Int32>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 871.5509965601407 > 208.5982247956943.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -485.0189768233603 (T) = (0 -874.3640227746596) / Math.Sqrt((8.076979175705105 / (33)) + (44.121325738566135 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.404647121576382 = (198.50943756459947 - 874.3640227746596) / 198.50943756459947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 2.05 ms 5.70 ms 2.78 0.01 False
Count - Duration of single invocation 461.06 μs 2.23 ms 4.84 0.01 False
Count - Duration of single invocation 1.15 ms 3.88 ms 3.38 0.02 False
Count - Duration of single invocation 1.40 ms 4.99 ms 3.55 0.01 False
Count - Duration of single invocation 440.32 μs 2.20 ms 5.00 0.01 False
Count - Duration of single invocation 1.97 ms 4.02 ms 2.03 0.01 False
Count - Duration of single invocation 1.22 ms 5.22 ms 4.29 0.01 False
Count - Duration of single invocation 3.66 ms 5.97 ms 1.63 0.01 False
Count - Duration of single invocation 3.13 ms 5.57 ms 1.78 0.01 False
Count - Duration of single invocation 406.58 μs 2.16 ms 5.31 0.01 False
Count - Duration of single invocation 519.34 μs 2.28 ms 4.39 0.01 False
Count - Duration of single invocation 1.13 ms 5.12 ms 4.54 0.01 False
Count - Duration of single invocation 4.18 ms 12.98 ms 3.10 0.01 False
Count - Duration of single invocation 955.89 μs 3.69 ms 3.86 0.01 False
Count - Duration of single invocation 520.09 μs 2.29 ms 4.40 0.01 False
Count - Duration of single invocation 1.26 ms 3.65 ms 2.89 0.01 False
Count - Duration of single invocation 1.34 ms 5.31 ms 3.96 0.01 False
Count - Duration of single invocation 490.87 μs 2.26 ms 4.60 0.01 False
Count - Duration of single invocation 1.08 ms 3.83 ms 3.53 0.01 False
Count - Duration of single invocation 460.16 μs 2.22 ms 4.83 0.01 False
Count - Duration of single invocation 3.44 ms 7.07 ms 2.06 0.01 False
Count - Duration of single invocation 2.85 ms 4.97 ms 1.74 0.01 False
Count - Duration of single invocation 4.07 ms 7.56 ms 1.86 0.01 False
Count - Duration of single invocation 427.66 μs 2.19 ms 5.12 0.01 False
Count - Duration of single invocation 469.93 μs 2.23 ms 4.75 0.01 False
Count - Duration of single invocation 443.68 μs 2.20 ms 4.96 0.01 False
Count - Duration of single invocation 1.26 ms 3.64 ms 2.89 0.01 False
Count - Duration of single invocation 406.42 μs 2.16 ms 5.31 0.01 False
Count - Duration of single invocation 439.30 μs 2.20 ms 5.01 0.01 False
Count - Duration of single invocation 452.42 μs 2.21 ms 4.89 0.01 False
Count - Duration of single invocation 1.22 ms 5.23 ms 4.28 0.01 False
Count - Duration of single invocation 2.06 ms 5.69 ms 2.76 0.01 False
Count - Duration of single invocation 1.12 ms 5.13 ms 4.57 0.01 False
Count - Duration of single invocation 438.96 μs 2.20 ms 5.01 0.01 False
Count - Duration of single invocation 1.08 ms 3.87 ms 3.58 0.06 False
Count - Duration of single invocation 1.23 ms 5.22 ms 4.25 0.01 False
Count - Duration of single invocation 10.10 ms 18.17 ms 1.80 0.01 False
Count - Duration of single invocation 407.07 μs 2.16 ms 5.31 0.01 False
Count - Duration of single invocation 2.74 ms 5.02 ms 1.83 0.01 False
Count - Duration of single invocation 1.41 ms 4.17 ms 2.96 0.02 False
Count - Duration of single invocation 1.94 ms 4.01 ms 2.07 0.01 False
Count - Duration of single invocation 1.06 ms 3.76 ms 3.55 0.03 False
Count - Duration of single invocation 437.27 μs 2.20 ms 5.04 0.01 False
Count - Duration of single invocation 1.26 ms 3.64 ms 2.89 0.01 False
Count - Duration of single invocation 481.29 μs 2.25 ms 4.68 0.01 False
Count - Duration of single invocation 434.01 μs 2.19 ms 5.05 0.01 False
Count - Duration of single invocation 2.23 ms 4.69 ms 2.10 0.01 False
Count - Duration of single invocation 1.54 ms 5.50 ms 3.57 0.01 False
Count - Duration of single invocation 2.81 ms 4.97 ms 1.77 0.01 False
Count - Duration of single invocation 1.26 ms 3.65 ms 2.90 0.01 False
Count - Duration of single invocation 439.49 μs 2.20 ms 5.00 0.01 False
Count - Duration of single invocation 478.38 μs 2.23 ms 4.66 0.01 False
Count - Duration of single invocation 559.59 μs 2.32 ms 4.15 0.01 False
Count - Duration of single invocation 2.03 ms 5.66 ms 2.79 0.02 False
Count - Duration of single invocation 1.86 ms 4.30 ms 2.31 0.01 False
Count - Duration of single invocation 427.66 μs 2.18 ms 5.11 0.01 False
Count - Duration of single invocation 2.77 ms 5.08 ms 1.83 0.01 False
Count - Duration of single invocation 1.19 ms 5.15 ms 4.32 0.01 False
Count - Duration of single invocation 460.62 μs 2.22 ms 4.82 0.01 False
Count - Duration of single invocation 491.78 μs 2.25 ms 4.57 0.01 False
Count - Duration of single invocation 1.28 ms 3.64 ms 2.85 0.01 False
Count - Duration of single invocation 1.22 ms 5.24 ms 4.30 0.01 False
Count - Duration of single invocation 446.14 μs 2.20 ms 4.94 0.01 False
Count - Duration of single invocation 1.18 ms 3.91 ms 3.32 0.01 False
Count - Duration of single invocation 3.11 ms 12.03 ms 3.87 0.01 False
Count - Duration of single invocation 1.26 ms 3.64 ms 2.89 0.01 False
Count - Duration of single invocation 1.00 ms 3.76 ms 3.76 0.01 False
Count - Duration of single invocation 968.34 μs 3.71 ms 3.83 0.02 False
Count - Duration of single invocation 498.13 μs 2.25 ms 4.52 0.01 False
Count - Duration of single invocation 611.48 μs 2.39 ms 3.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.696468175595239 > 2.0974717428605767.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -393.9405577340021 (T) = (0 -5660317.588236119) / Math.Sqrt((721949331.7650745 / (33)) + (1671588764.2885296 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8252532276323523 = (2003472.656141211 - 5660317.588236119) / 2003472.656141211 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2320346446428574 > 487.31472525874557.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2543.9540593697093 (T) = (0 -2233153.8973204847) / Math.Sqrt((9092933.300382944 / (33)) + (5387495.529506467 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.7962885023049178 = (465600.41086921986 - 2233153.8973204847) / 465600.41086921986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8828925535714287 > 1.1795697784375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -571.6050678469624 (T) = (0 -3880815.8898874456) / Math.Sqrt((738098542.7035463 / (33)) + (18043149.7700051 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.4165544026025656 = (1135885.875820162 - 3880815.8898874456) / 1135885.875820162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.986065193509615 > 1.4696713766512786.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -637.1086795623061 (T) = (0 -4998714.840513392) / Math.Sqrt((403196565.71069735 / (33)) + (513571993.9838905 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.579905346440385 = (1396325.9798150181 - 4998714.840513392) / 1396325.9798150181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.200692036658654 > 462.5214669885262.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -4257.601538475274 (T) = (0 -2201468.6777158054) / Math.Sqrt((1515194.3051799012 / (33)) + (3252164.5625656312 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.993485218372337 = (440868.1675107452 - 2201468.6777158054) / 440868.1675107452 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.015384828125 > 2.0208002742187503.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -377.6698681726237 (T) = (0 -4025079.0389381265) / Math.Sqrt((288334167.3754637 / (33)) + (572190886.9277761 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.0844775395656154 = (1930977.4092250057 - 4025079.0389381265) / 1930977.4092250057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.223069575694444 > 1.2838343344110572.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -4174.323292292245 (T) = (0 -5227664.216089627) / Math.Sqrt((4947547.777513549 / (33)) + (20041885.910509583 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.2775224484730305 = (1222124.320576219 - 5227664.216089627) / 1222124.320576219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.971914325783972 > 3.8143308640130096.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -441.25158918586055 (T) = (0 -5954352.970959382) / Math.Sqrt((365020832.82854635 / (33)) + (432401829.8042261 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.6392594762177578 = (3632343.175284113 - 5954352.970959382) / 3632343.175284113 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.574354292424243 > 3.286287765080331.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -475.0492704339753 (T) = (0 -5582896.114889113) / Math.Sqrt((469584867.83023995 / (33)) + (317112936.7942821 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.7775258101168392 = (3140824.219324355 - 5582896.114889113) / 3140824.219324355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1596457630208334 > 427.5335813766892.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3691.835095481671 (T) = (0 -2159758.918175069) / Math.Sqrt((5302689.302217419 / (33)) + (1676250.586525655 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.294173317506669 = (407950.1725100727 - 2159758.918175069) / 407950.1725100727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2808294423076925 > 544.1114737650241.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2445.2889484309926 (T) = (0 -2279331.019900915) / Math.Sqrt((5249300.067694316 / (33)) + (9371368.04091112 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.410092377629016 = (516844.28005708684 - 2279331.019900915) / 516844.28005708684 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.118866329927885 > 1.1830564841964282.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -5254.236364891649 (T) = (0 -5122910.408603912) / Math.Sqrt((3471432.3742811894 / (33)) + (12302170.354990823 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.545291621614751 = (1127080.6881218234 - 5122910.408603912) / 1127080.6881218234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.97757182330827 > 4.406451012529223.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -428.3916832218934 (T) = (0 -12806282.548930349) / Math.Sqrt((2864027697.623309 / (33)) + (8235729528.213734 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.0487234427248255 = (4200539.271441627 - 12806282.548930349) / 4200539.271441627 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6939198913461535 > 1003.9319787365669.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1834.9298735923662 (T) = (0 -3699701.618009475) / Math.Sqrt((44684490.75109883 / (33)) + (23410028.050647184 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.916770972059806 = (944579.5131758302 - 3699701.618009475) / 944579.5131758302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.287678893494898 > 545.4048071157852.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2827.437326560269 (T) = (0 -2290274.131476233) / Math.Sqrt((7517852.084919028 / (33)) + (4266590.3172063185 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.4025609153127636 = (520214.06984065066 - 2290274.131476233) / 520214.06984065066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6464097404166664 > 1.3230709200240385.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3341.2873310813543 (T) = (0 -3642150.713987038) / Math.Sqrt((987543.6706803901 / (33)) + (12432470.09363053 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8895658664578951 = (1260449.10630526 - 3642150.713987038) / 1260449.10630526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.307832046808511 > 1.4107914244270834.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1748.0861682026118 (T) = (0 -5305664.763974035) / Math.Sqrt((97058799.19922774 / (33)) + (56958002.25983748 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.942927579436389 = (1345615.5755040366 - 5305664.763974035) / 1345615.5755040366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.259203825 > 515.8044857998605.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3809.431152318795 (T) = (0 -2259752.593067527) / Math.Sqrt((6459427.28961568 / (33)) + (507924.0461091555 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.590570702999877 = (492259.620702674 - 2259752.593067527) / 492259.620702674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8304985072115385 > 1.146704660720524.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -145.07820196023036 (T) = (0 -3824198.235694469) / Math.Sqrt((11547281108.420046 / (33)) + (19810189.22442205 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.4533123169850737 = (1107400.0509265256 - 3824198.235694469) / 1107400.0509265256 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2216643618181817 > 483.21182443006893.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -5766.683718335623 (T) = (0 -2219894.331485233) / Math.Sqrt((1087328.5053728276 / (33)) + (1563503.0629564151 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.820645169482827 = (460497.3511716461 - 2219894.331485233) / 460497.3511716461 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.073479726190475 > 3.606438598875.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -306.941005975371 (T) = (0 -7032648.0534518175) / Math.Sqrt((2734596584.419992 / (33)) + (1399394615.5820959 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.0419584454175368 = (3444070.1128047644 - 7032648.0534518175) / 3444070.1128047644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.966346668229167 > 2.9934572381610582.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -185.7741496410309 (T) = (0 -4945670.666299156) / Math.Sqrt((1910707652.174197 / (33)) + (1911034662.8169215 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.7561928372708628 = (2816131.8969873297 - 4945670.666299156) / 2816131.8969873297 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.555870825174825 > 4.251998326310153.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -280.8207510703375 (T) = (0 -7531790.883771105) / Math.Sqrt((3579234819.1588435 / (33)) + (1276346874.8284857 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.879687626566907 = (4006937.5237242444 - 7531790.883771105) / 4006937.5237242444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1908277400841345 > 450.19711154983116.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2608.964912126498 (T) = (0 -2190950.1551796924) / Math.Sqrt((12312902.27986392 / (33)) + (2169079.7191369846 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.117572373418707 = (428122.9448868674 - 2190950.1551796924) / 428122.9448868674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2325745337837835 > 493.326616631434.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3271.0416139528234 (T) = (0 -2229835.040449142) / Math.Sqrt((1843668.2582376334 / (33)) + (6067677.035494269 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.7379578512338396 = (470632.09730083554 - 2229835.040449142) / 470632.09730083554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1988761670918366 > 464.9808012231924.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3900.975617082843 (T) = (0 -2199422.877407416) / Math.Sqrt((5069718.769739982 / (33)) + (1280712.3406424508 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.9725651613009476 = (442311.52454762236 - 2199422.877407416) / 442311.52454762236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6432422874999997 > 1.3235714653185096.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3676.9373698507866 (T) = (0 -3642209.386150852) / Math.Sqrt((1090346.2144346833 / (33)) + (10050289.305279199 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8896243212401853 = (1260443.9128570415 - 3642209.386150852) / 1260443.9128570415 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.15893252734375 > 428.06406272804054.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3684.7471531204255 (T) = (0 -2159752.791590446) / Math.Sqrt((4401119.985874813 / (33)) + (2407433.012061545 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.290973576027248 = (408195.7243891788 - 2159752.791590446) / 408195.7243891788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.20037062202381 > 461.48929207732374.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3716.362554155657 (T) = (0 -2201153.1270621065) / Math.Sqrt((6399054.627502539 / (33)) + (799254.0831568817 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.006100488540813 = (439694.15558090457 - 2201153.1270621065) / 439694.15558090457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.212947523809524 > 475.6829616827065.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3901.0618284643783 (T) = (0 -2213638.4940426014) / Math.Sqrt((3040814.5304445596 / (33)) + (2907207.6454165755 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.899198649199536 = (451836.8518092812 - 2213638.4940426014) / 451836.8518092812 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.229044865793781 > 1.2849810220110296.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3041.2953739151862 (T) = (0 -5233951.263008009) / Math.Sqrt((8304686.050428398 / (33)) + (38638465.06305597 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.2733126665606442 = (1224799.5106851214 - 5233951.263008009) / 1224799.5106851214 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.6855626905187835 > 2.1201057417140405.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -499.72859065293045 (T) = (0 -5686806.304460835) / Math.Sqrt((508362844.3854991 / (33)) + (989160540.524219 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.7967947269435467 = (2033329.8864145144 - 5686806.304460835) / 2033329.8864145144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.126279876201923 > 1.1788680046093754.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3031.287633518747 (T) = (0 -5129755.951007693) / Math.Sqrt((37840019.347904555 / (33)) + (15596247.428249002 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.564956402882968 = (1123725.0694810643 - 5129755.951007693) / 1123725.0694810643 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.197409275510204 > 456.1511258198116.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1627.2008291344969 (T) = (0 -2198070.0035421588) / Math.Sqrt((34704868.08592652 / (33)) + (3125438.3974207193 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.034795673317962 = (436575.81084986054 - 2198070.0035421588) / 436575.81084986054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.872194886160714 > 1.1372312196714625.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -384.82438491945027 (T) = (0 -3872173.7826302303) / Math.Sqrt((1620684693.2481074 / (33)) + (48250282.93934574 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.4423745420504415 = (1124855.4552474061 - 3872173.7826302303) / 1124855.4552474061 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.223940929078013 > 1.2875712243025474.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3443.3312830069 (T) = (0 -5231376.912488561) / Math.Sqrt((9286581.111314772 / (33)) + (27850806.34018566 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.2644473501093323 = (1226742.056588983 - 5231376.912488561) / 1226742.056588983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.169739917159763 > 10.589034723000003.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -288.2085738358391 (T) = (0 -18154102.719468106) / Math.Sqrt((10450778266.972193 / (33)) + (12228841309.924677 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.8030267234470052 = (10068682.001984589 - 18154102.719468106) / 10068682.001984589 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1599663526785715 > 429.91210503589537.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3036.8278274064414 (T) = (0 -2159700.4430758995) / Math.Sqrt((9163581.5106431 / (33)) + (1418099.2200113172 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.276567421685279 = (409300.26482749166 - 2159700.4430758995) / 409300.26482749166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.0215144140625 > 2.8733215946354163.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -740.3600086886076 (T) = (0 -5036027.861262305) / Math.Sqrt((17000861.47988731 / (33)) + (237302653.15296605 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.8399341437026342 = (2737069.627463914 - 5036027.861262305) / 2737069.627463914 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.174221978794643 > 1.4829201323579546.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1065.9135304064782 (T) = (0 -4176713.4281880674) / Math.Sqrt((142952704.67366585 / (33)) + (62003593.032385245 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.9533317615691923 = (1414237.8050913103 - 4176713.4281880674) / 1414237.8050913103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.007472858333333 > 2.0271836481933594.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -335.4199514298259 (T) = (0 -4023052.1692448575) / Math.Sqrt((460746703.81819516 / (33)) + (645123406.6408089 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.079719115944622 = (1934420.9217491185 - 4023052.1692448575) / 1934420.9217491185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.762625705641025 > 1.1146460064616563.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -694.4286713147708 (T) = (0 -3758499.7685432984) / Math.Sqrt((464386308.10795873 / (33)) + (26821013.547775503 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5467870753930217 = (1059691.4020069323 - 3758499.7685432984) / 1059691.4020069323 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.202370947619048 > 459.5886024913194.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -4176.724903543374 (T) = (0 -2202815.6778797098) / Math.Sqrt((1932538.1188246838 / (33)) + (3117290.1135768043 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.024782943694217 = (438390.2155702274 - 2202815.6778797098) / 438390.2155702274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.641052740833333 > 1.3236648969868714.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2044.6946140141695 (T) = (0 -3641968.030042) / Math.Sqrt((26027802.763331134 / (33)) + (14725395.20575871 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8863742276894269 = (1261779.5693656239 - 3641968.030042) / 1261779.5693656239 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2529219693877556 > 506.68106462402346.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2851.3614555677646 (T) = (0 -2252461.208352579) / Math.Sqrt((6266523.056005198 / (33)) + (5083025.080590696 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.669966380553001 = (482329.212846678 - 2252461.208352579) / 482329.212846678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1909056864583336 > 454.2507361071048.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3564.6376016930476 (T) = (0 -2192665.8977192785) / Math.Sqrt((5728913.152149775 / (33)) + (1831321.685815962 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.0788540049796485 = (431724.537773569 - 2192665.8977192785) / 431724.537773569 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.689357530133928 > 2.3443316183293272.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -827.8091313988128 (T) = (0 -4693600.439059418) / Math.Sqrt((213244281.75589865 / (33)) + (60550479.9867322 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.0961022577857693 = (2239203.9422816765 - 4693600.439059418) / 2239203.9422816765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.495669721726191 > 1.6133809990553978.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2224.7498211302377 (T) = (0 -5498492.522075468) / Math.Sqrt((19372042.815531638 / (33)) + (67120356.08008865 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.5743419933514287 = (1538323.0066689528 - 5498492.522075468) / 1538323.0066689528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.9742949953124995 > 2.965414394166667.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -226.7496620595351 (T) = (0 -4977243.649888832) / Math.Sqrt((1153816113.349504 / (33)) + (1418171026.466462 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.7575128583836007 = (2831981.357147193 - 4977243.649888832) / 2831981.357147193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.653977035416666 > 1.3247397518629809.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2353.076015784566 (T) = (0 -3643657.837797443) / Math.Sqrt((855523.6285724675 / (33)) + (25989157.403247617 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8900493180503937 = (1260759.743800991 - 3643657.837797443) / 1260759.743800991 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1977687321428574 > 460.59803864583336.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3692.45794037642 (T) = (0 -2197849.2574005066) / Math.Sqrt((4649151.170429157 / (33)) + (2233636.9411916584 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.001934764224544 = (439399.824467971 - 2197849.2574005066) / 439399.824467971 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.230618846497253 > 502.37227404785165.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2929.5258810033015 (T) = (0 -2231284.2646627696) / Math.Sqrt((10655355.379018461 / (33)) + (909227.1193976955 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.6601470321627168 = (478801.25868630764 - 2231284.2646627696) / 478801.25868630764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3236956779874207 > 586.2773093508808.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2742.613128697962 (T) = (0 -2325941.8454486537) / Math.Sqrt((2950498.811625617 / (33)) + (8474007.450889548 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.1651489009217144 = (558429.4585324287 - 2325941.8454486537) / 558429.4585324287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.663959911458333 > 2.1294122102779447.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -636.484772025513 (T) = (0 -5636653.153537234) / Math.Sqrt((333070726.39375293 / (33)) + (564284052.3572422 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.7527688040917142 = (2047630.424014874 - 5636653.153537234) / 2047630.424014874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.303007951041667 > 1.9721331294097222.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1114.9475226548575 (T) = (0 -4309533.312071622) / Math.Sqrt((69595831.80005296 / (33)) + (68695376.73172605 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.2931812569031413 = (1879281.5871395583 - 4309533.312071622) / 1879281.5871395583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.184143179129464 > 447.24517943412167.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2852.2109649339145 (T) = (0 -2183325.473592143) / Math.Sqrt((11420094.626221618 / (33)) + (855920.5549464535 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -4.107698772078222 = (427457.7595545618 - 2183325.473592143) / 427457.7595545618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.0794892109375 > 2.9084186699559296.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -282.4400318914808 (T) = (0 -5098584.162251472) / Math.Sqrt((1953510015.2253995 / (33)) + (216221990.64514092 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.8354290881460282 = (2777870.414705896 - 5098584.162251472) / 2777870.414705896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.154616709821428 > 1.2526000800915311.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3196.3786291027996 (T) = (0 -5153156.083658067) / Math.Sqrt((25623258.06918742 / (33)) + (19696310.510993842 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.3148905257204975 = (1194272.7290393065 - 5153156.083658067) / 1194272.7290393065 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2212847783783785 > 483.7782669182132.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3903.7983052567997 (T) = (0 -2221365.0175198936) / Math.Sqrt((4465235.945304169 / (33)) + (1768934.7711074322 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.8186893935838584 = (460989.4591831686 - 2221365.0175198936) / 460989.4591831686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.246446873376623 > 514.1054991713788.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2620.6694719038487 (T) = (0 -2249093.7731332034) / Math.Sqrt((7528194.967654996 / (33)) + (5807007.06810793 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.6067287470336495 = (488219.27589755156 - 2249093.7731332034) / 488219.27589755156 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.64312951125 > 1.3238690267612798.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1095.2231148608962 (T) = (0 -3642894.7707449277) / Math.Sqrt((131335444.50778127 / (33)) + (19311703.27027614 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.8847737285597204 = (1262800.8688098092 - 3642894.7707449277) / 1262800.8688098092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.236124792467948 > 1.2804693926682693.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -386.76957323532076 (T) = (0 -5240106.939051641) / Math.Sqrt((12887502.58537431 / (33)) + (2798162954.5326147 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.2935755854795565 = (1220452.9382860195 - 5240106.939051641) / 1220452.9382860195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.204963037627551 > 467.7908398493304.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -7471.068704182536 (T) = (0 -2205821.831805771) / Math.Sqrt((1201081.8131611228 / (33)) + (496478.29631979665 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.9470560060810826 = (445885.7609645622 - 2205821.831805771) / 445885.7609645622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.91174990625 > 1.23404193640625.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1447.0157991499489 (T) = (0 -3914673.4688792177) / Math.Sqrt((99052521.12475507 / (33)) + (14863264.402185423 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.319288438487038 = (1179371.2843658628 - 3914673.4688792177) / 1179371.2843658628 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.034934542857144 > 3.2650656620937495.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -344.01369240027753 (T) = (0 -11869242.890622612) / Math.Sqrt((657534755.3280046 / (33)) + (16292576029.945503 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.8020505747189035 = (3121800.3699227856 - 11869242.890622612) / 3121800.3699227856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6421680941666668 > 1.3232230031249996.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -3750.8055430556196 (T) = (0 -3642495.5068297405) / Math.Sqrt((475124.595450447 / (33)) + (10115681.428932745 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.890793741904917 = (1260032.9985595867 - 3642495.5068297405) / 1260032.9985595867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.757598894642857 > 1.0548234960471756.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -1104.099724844498 (T) = (0 -3763591.9789282186) / Math.Sqrt((168990637.72298163 / (33)) + (29211407.59438132 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.746471373652964 = (1004569.794766258 - 3763591.9789282186) / 1004569.794766258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7126148274038457 > 1024.6317482910158.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -753.9254136566373 (T) = (0 -3712649.000999665) / Math.Sqrt((411587885.7485038 / (33)) + (17347370.2891951 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.7892702100264906 = (979779.4285495703 - 3712649.000999665) / 979779.4285495703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2505660665137612 > 520.5459291493056.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2940.0300362270814 (T) = (0 -2246732.88014169) / Math.Sqrt((8380861.957808117 / (33)) + (2602131.36974409 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -3.517322224289226 = (497359.44628019095 - 2246732.88014169) / 497359.44628019095 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3868426747572817 > 641.6045950190567.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -2390.472252352795 (T) = (0 -2385042.189447099) / Math.Sqrt((7309758.172507058 / (33)) + (8545691.926916335 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -2.8976557321998744 = (611917.0992305567 - 2385042.189447099) / 611917.0992305567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FourArguments_EnumerableArgument - Duration of single invocation 2.63 μs 2.90 μs 1.10 0.05 False
TwoArguments_DefineMessage - Duration of single invocation 232.93 ns 330.55 ns 1.42 0.01 False
TwoArguments - Duration of single invocation 370.47 ns 473.63 ns 1.28 0.01 False
FourArguments_DefineMessage - Duration of single invocation 409.66 ns 642.21 ns 1.57 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_EnumerableArgument

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8951624587155966 > 2.7680571293790637.
IsChangePoint: Marked as a change because one of 7/18/2022 2:17:16 AM, 7/28/2022 6:38:38 AM, 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -12.238702567579999 (T) = (0 -2854.720039286341) / Math.Sqrt((8883.820470212948 / (33)) + (862.832214162087 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.08055735808373579 = (2641.8958863497182 - 2854.720039286341) / 2641.8958863497182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 330.54894506143995 > 243.9375863571776.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -292.14538338004576 (T) = (0 -332.34843467045533) / Math.Sqrt((0.5787877891386953 / (33)) + (2.561219530970255 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.427448324103793 = (232.82694655802442 - 332.34843467045533) / 232.82694655802442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 473.6299439541489 > 391.3592038404241.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -102.47225725292788 (T) = (0 -475.2227154620775) / Math.Sqrt((7.580113334045035 / (33)) + (20.30884774733179 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.276800528970316 = (372.1980878605399 - 475.2227154620775) / 372.1980878605399 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 642.2142956775618 > 429.2300920609582.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -185.030653440433 (T) = (0 -644.467164390659) / Math.Sqrt((25.714028446707246 / (33)) + (21.49338508118893 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.5719092558254368 = (409.9900563612612 - 644.467164390659) / 409.9900563612612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in BenchmarksGame.RegexRedux_1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 55.92 ms 137.21 ms 2.45 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.RegexRedux_1*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

BenchmarksGame.RegexRedux_1.RunBench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 137.20576946666668 > 58.693761037499996.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -421.2523086095481 (T) = (0 -136713410.05465272) / Math.Sqrt((221109230361.22626 / (33)) + (773791849915.0381 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -1.4293497943657203 = (56275720.512429245 - 136713410.05465272) / 56275720.512429245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf - Duration of single invocation 14.23 ns 22.17 ns 1.56 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.170126842729907 > 14.904405190652941.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -59.80695994517736 (T) = (0 -22.19057652223892) / Math.Sqrt((0.5605794666118827 / (33)) + (0.0024582034217767055 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.5438035329537209 = (14.373964075456055 - 22.19057652223892) / 14.373964075456055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 1.14 ms 1.21 ms 1.06 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2077319302967566 > 1.196875435851335.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -34.781925642226845 (T) = (0 -1210588.5143807826) / Math.Sqrt((44399559.70550947 / (33)) + (66464784.89156802 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.06016763677118138 = (1141884.0496468267 - 1210588.5143807826) / 1141884.0496468267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 34.21 μs 40.79 μs 1.19 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndRemove<Int32>.List(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.785912865470024 > 35.63672533275105.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -77.56562714620968 (T) = (0 -40773.01344088483) / Math.Sqrt((97922.81269425062 / (33)) + (123923.60092671975 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.20090870595568913 = (33951.8010309014 - 40773.01344088483) / 33951.8010309014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 232.48 ns 331.76 ns 1.43 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Logging.Formatting*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Microsoft.Extensions.Logging.Formatting.TwoArguments

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 331.7611553708069 > 244.6947134006263.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -157.39906110010116 (T) = (0 -332.0330957787554) / Math.Sqrt((8.558779933372081 / (33)) + (3.3828973833291043 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.42013176792876916 = (233.80442806586768 - 332.0330957787554) / 233.80442806586768 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@jozkee
Copy link
Member

jozkee commented Oct 13, 2022

@DrewScoggins we noticed during the RC2 perf report that this benchmark regressed on all configs. Should we transfer this issue to dotnet/runtime?

Thanks!

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 0.86 +0 ubuntu 18.04 Arm64 Unknown processor
Same 0.90 -11 Windows 11 Arm64 Unknown processor
Same 0.92 +168 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Same 0.94 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 0.83 +34 macOS Monterey 12.6 Arm64 Apple M1
Slower 0.85 +34 macOS Monterey 12.6 Arm64 Apple M1 Max
Same 1.08 +0 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower 0.73 -242 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 0.75 +0 Windows 11 X64 AMD Ryzen 9 5900X
Slower 0.67 +16 Windows 11 X64 AMD Ryzen 9 7950X
Slower 0.86 +0 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower 0.68 +87 debian 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower 0.51 +17 ubuntu 18.04 X64 AMD Ryzen 9 5900X
Slower 0.65 +87 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 0.50 +17 ubuntu 20.04 X64 AMD Ryzen 9 5900X
Slower 0.73 +0 ubuntu 20.04 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) bimodal
Slower 0.70 +68 ubuntu 20.04 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower 0.65 +68 macOS Big Sur 11.7 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower 0.65 -138 macOS Monterey 12.6 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

@jozkee jozkee mentioned this issue Oct 18, 2022
Open
26 tasks
@kotlarmilos
Copy link
Member

Real regression - investigation needed

@kotlarmilos kotlarmilos removed the untriaged New issue has not been triaged by the area owner label Oct 18, 2022
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Nov 4, 2022
@kotlarmilos kotlarmilos transferred this issue from dotnet/perf-autofiling-issues Nov 4, 2022
@kotlarmilos kotlarmilos added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark and removed untriaged New issue has not been triaged by the area owner labels Nov 4, 2022
@jeffschwMSFT jeffschwMSFT added area-System.Memory untriaged New issue has not been triaged by the area owner labels Nov 7, 2022
@ghost
Copy link

ghost commented Nov 7, 2022

Tagging subscribers to this area: @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 969.31 μs 1.10 ms 1.13 0.07 False
ImmutableArray - Duration of single invocation 996.73 μs 1.11 ms 1.12 0.05 False
ICollection - Duration of single invocation 967.59 μs 1.06 ms 1.10 0.05 False
List - Duration of single invocation 954.73 μs 1.06 ms 1.11 0.06 False
Array - Duration of single invocation 974.99 μs 1.07 ms 1.10 0.06 False
Stack - Duration of single invocation 1.00 ms 1.24 ms 1.23 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.095966200892857 > 1010.798569984976.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -26.205242060142314 (T) = (0 -1082176.5961467223) / Math.Sqrt((134557800.542629 / (33)) + (378753722.89546967 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11677096832909976 = (969022.8586134029 - 1082176.5961467223) / 969022.8586134029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1139794651785715 > 1.060484347207031.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -27.791974606900073 (T) = (0 -1115382.3246755877) / Math.Sqrt((180256514.8127183 / (33)) + (235012253.979763 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10483220306009597 = (1009549.0714212264 - 1115382.3246755877) / 1009549.0714212264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0617160416666667 > 1018.8446187011718.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -16.056018293701424 (T) = (0 -1087710.4426824627) / Math.Sqrt((185033592.69745567 / (33)) + (1162193806.342326 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11694145311763579 = (973829.4157195234 - 1087710.4426824627) / 973829.4157195234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0611418856944443 > 1018.2846668750002.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -35.482093782068404 (T) = (0 -1074002.2283184354) / Math.Sqrt((69967824.32398583 / (33)) + (179800464.4794185 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11025724102303357 = (967345.3940537317 - 1074002.2283184354) / 967345.3940537317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0702191419642857 > 1023.6990125195313.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -30.120564854575687 (T) = (0 -1086000.919079096) / Math.Sqrt((291634961.6495431 / (33)) + (86090478.21488371 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10701630982952783 = (981016.1868765348 - 1086000.919079096) / 981016.1868765348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2352883976648354 > 1024.4168104882813.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -60.858087292552526 (T) = (0 -1236941.1502558445) / Math.Sqrt((337610251.7351471 / (33)) + (203005793.13338426 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2641636960719056 = (978465.9645735368 - 1236941.1502558445) / 978465.9645735368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 4.81 μs 5.38 μs 1.12 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.378923124141483 > 5.053520181621476.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -51.38447919574725 (T) = (0 -5367.488921550735) / Math.Sqrt((2937.7007674825045 / (32)) + (472.35950356068855 / (26))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (26) - 2, .025) and -0.11159539603981007 = (4828.635437563927 - 5367.488921550735) / 4828.635437563927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 1.89 ms 2.38 ms 1.26 0.06 False
ImmutableArray - Duration of single invocation 1.93 ms 2.14 ms 1.11 0.03 False
Array - Duration of single invocation 1.90 ms 2.11 ms 1.11 0.03 False
List - Duration of single invocation 1.89 ms 2.10 ms 1.11 0.05 False
Queue - Duration of single invocation 1.89 ms 2.23 ms 1.18 0.06 False
ICollection - Duration of single invocation 1.87 ms 2.10 ms 1.12 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3842154708333334 > 1.9866159997796478.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -63.97195320199993 (T) = (0 -2394094.2410446103) / Math.Sqrt((1695724547.8362103 / (33)) + (190835740.57041073 / (25))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (25) - 2, .025) and -0.2583031159565782 = (1902637.1394022887 - 2394094.2410446103) / 1902637.1394022887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1394566171875 > 2.0256352184960935.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -31.077029944721016 (T) = (0 -2151332.1590478206) / Math.Sqrt((360162614.46956015 / (33)) + (997867569.4428535 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11286796194600314 = (1933142.3247066252 - 2151332.1590478206) / 1933142.3247066252 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1087881843750003 > 1.9809466521310766.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -45.259498036275815 (T) = (0 -2113783.74757238) / Math.Sqrt((353745729.22009766 / (33)) + (319361059.55712956 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11444507736727372 = (1896714.1499390076 - 2113783.74757238) / 1896714.1499390076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1026052706473215 > 1.9608499316145835.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -68.80938666641624 (T) = (0 -2099614.6035013716) / Math.Sqrt((211620946.8627083 / (33)) + (92498111.32462098 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11542629051873408 = (1882342.7611025164 - 2099614.6035013716) / 1882342.7611025164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2304079871651785 > 2.008788108359375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -23.59913129326589 (T) = (0 -2147578.562214543) / Math.Sqrt((1116474284.4023762 / (33)) + (1587306697.1320746 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11986933064211758 = (1917704.5959309835 - 2147578.562214543) / 1917704.5959309835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0979382890625 > 1.9806727393359376.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -42.85148744252998 (T) = (0 -2103699.1899045617) / Math.Sqrt((347858606.2090312 / (33)) + (396049417.258107 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11533974964320046 = (1886151.005178054 - 2103699.1899045617) / 1886151.005178054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 832.17 ns 895.38 ns 1.08 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 895.3778104156972 > 849.9629668568364.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -29.21081282954959 (T) = (0 -892.9335262280066) / Math.Sqrt((168.6694102042165 / (33)) + (82.54061160444941 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10395462812604744 = (808.8498417219853 - 892.9335262280066) / 808.8498417219853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 1.87 μs 2.49 μs 1.33 0.06 False
IndexOfValue - Duration of single invocation 1.98 μs 2.11 μs 1.07 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.486075723145284 > 1.961551330630418.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -32.74534469892047 (T) = (0 -2378.1927078714825) / Math.Sqrt((666.1354551089895 / (33)) + (5810.079235983002 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2737630503998014 = (1867.0605236390152 - 2378.1927078714825) / 1867.0605236390152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1059525350629564 > 1.9711226404728943.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -26.75366215584882 (T) = (0 -2108.271736496125) / Math.Sqrt((824.5871390603775 / (33)) + (1462.8924148464291 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.1291609019154783 = (1867.1136530849672 - 2108.271736496125) / 1867.1136530849672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: jeffhandley, adamsitnik, kotlarmilos
Labels:

area-System.Memory, tenet-performance, tenet-performance-benchmarks, refs/heads/main, ubuntu 18.04, Regression, x64
Milestone: -

@kotlarmilos kotlarmilos mentioned this issue Nov 8, 2022
Closed
adamsitnik added a commit to adamsitnik/performance that referenced this issue Nov 8, 2022
@adamsitnik
use 7.0.0 Microsoft.Extensions* when targeting net7.0, so .NET 6 vs 7…
35eefc6 
… diff shows the actual difference for Extensions benchmarks, fixes dotnet/runtime#77900
@adamsitnik adamsitnik mentioned this issue Nov 8, 2022
Merged
@adamsitnik
Copy link
Member

adamsitnik commented Nov 8, 2022 •
edited
Loading

we noticed during the RC2 perf report that this benchmark regressed on all configs. Should we transfer this issue to dotnet/runtime?

It was not an actual regression; we were benchmarking an old version of Microsoft.Extensions when targetting net7.0. I've sent a fix: dotnet/performance#2703

Before:

BenchmarkDotNet=v0.13.2.1940-nightly, OS=Windows 11 (10.0.22621.674)
AMD Ryzen Threadripper PRO 3945WX 12-Cores, 1 CPU, 24 logical and 12 physical cores
.NET SDK=7.0.100-rc.1.22431.12
  [Host]     : .NET 6.0.10 (6.0.1022.47605), X64 RyuJIT AVX2
  Job-EOWPLY : .NET 6.0.10 (6.0.1022.47605), X64 RyuJIT AVX2
  Job-RERDVV : .NET 7.0.0 (7.0.22.42610), X64 RyuJIT AVX2
Method Runtime Mean Error StdDev Median Min Max Ratio Gen0 Gen1 Allocated Alloc Ratio
AddChainedConfigurationNoDelimiter .NET 6.0 1,134.3 us 9.98 us 8.85 us 1,133.6 us 1,113.7 us 1,148.5 us 1.00 118.7500 6.2500 1010.97 KB 1.00
AddChainedConfigurationNoDelimiter .NET 7.0 975.2 us 10.67 us 8.91 us 974.1 us 959.3 us 995.5 us 0.86 118.7500 6.2500 1010.97 KB 1.00
AddChainedConfigurationEmpty .NET 6.0 36,984.2 us 213.40 us 189.17 us 36,995.7 us 36,652.1 us 37,313.1 us 1.00 - - 1010.43 KB 1.00
AddChainedConfigurationEmpty .NET 7.0 40,250.9 us 203.40 us 169.85 us 40,188.6 us 40,022.1 us 40,542.8 us 1.09 111.1111 - 1010.2 KB 1.00
AddChainedConfigurationWithSplitting .NET 6.0 429.2 us 3.60 us 2.81 us 430.1 us 423.6 us 432.1 us 1.00 57.2917 5.2083 471.68 KB 1.00
AddChainedConfigurationWithSplitting .NET 7.0 511.2 us 4.75 us 4.44 us 510.0 us 506.2 us 520.1 us 1.19 56.4516 6.0484 471.68 KB 1.00
AddChainedConfigurationWithCommonPaths .NET 6.0 417.4 us 3.84 us 3.59 us 418.1 us 411.7 us 424.6 us 1.00 57.5658 4.9342 471.68 KB 1.00
AddChainedConfigurationWithCommonPaths .NET 7.0 470.3 us 4.09 us 3.41 us 469.2 us 466.3 us 478.8 us 1.13 56.9853 5.5147 471.68 KB 1.00

After:

BenchmarkDotNet=v0.13.2.1950-nightly, OS=Windows 11 (10.0.22621.674)
AMD Ryzen Threadripper PRO 3945WX 12-Cores, 1 CPU, 24 logical and 12 physical cores
.NET SDK=8.0.100-alpha.1.22558.1
  [Host]     : .NET 6.0.10 (6.0.1022.47605), X64 RyuJIT AVX2
  Job-REUZGI : .NET 6.0.10 (6.0.1022.47605), X64 RyuJIT AVX2
  Job-XTHPSX : .NET 7.0.0 (7.0.22.42610), X64 RyuJIT AVX2
Method Runtime Mean Error StdDev Median Min Max Ratio RatioSD Gen0 Gen1 Allocated Alloc Ratio
AddChainedConfigurationNoDelimiter .NET 6.0 894.7 us 15.55 us 14.55 us 896.9 us 865.0 us 917.0 us 1.00 0.00 119.3182 5.6818 1010.97 KB 1.00
AddChainedConfigurationNoDelimiter .NET 7.0 634.1 us 2.14 us 1.78 us 633.9 us 630.5 us 636.8 us 0.71 0.01 17.0455 - 143.22 KB 0.14
AddChainedConfigurationEmpty .NET 6.0 36,917.5 us 134.65 us 112.44 us 36,885.4 us 36,785.7 us 37,166.4 us 1.00 0.00 - - 1010.43 KB 1.00
AddChainedConfigurationEmpty .NET 7.0 28,867.2 us 172.48 us 152.90 us 28,843.3 us 28,663.4 us 29,128.1 us 0.78 0.00 - - 143.26 KB 0.14
AddChainedConfigurationWithSplitting .NET 6.0 455.7 us 11.41 us 13.14 us 457.3 us 435.0 us 482.9 us 1.00 0.00 57.6923 4.8077 471.68 KB 1.00
AddChainedConfigurationWithSplitting .NET 7.0 318.4 us 1.38 us 1.22 us 318.8 us 316.6 us 320.2 us 0.70 0.02 3.7500 - 40.43 KB 0.09
AddChainedConfigurationWithCommonPaths .NET 6.0 470.0 us 7.93 us 7.41 us 472.0 us 456.1 us 480.7 us 1.00 0.00 56.9853 5.5147 471.68 KB 1.00
AddChainedConfigurationWithCommonPaths .NET 7.0 353.1 us 4.01 us 3.55 us 353.1 us 348.0 us 360.7 us 0.75 0.01 4.1667 - 40.43 KB 0.09

@adamsitnik adamsitnik removed the untriaged New issue has not been triaged by the area owner label Nov 8, 2022
@dakersnar
Copy link
Contributor

Sure enough, this benchmark no longer shows up as regressed in the GA vs GA report.

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Result Ratio Alloc Delta Operating System Bit Processor Name Modality
Faster 1.66 -887829 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Faster 1.34 -887772 macOS 13.0 Arm64 Apple M1
Same 0.98 -887744 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 1.09 -887796 Windows 10 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Faster 1.66 -887817 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 1.05 -887751 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Faster 1.20 -887762 Windows 11 X64 11th Gen Intel Core i9-11900 2.50GHz
Same 1.02 -888033 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Faster 1.64 -444254 raspbian 11 Arm ARMv7 Processor rev 3 (v7l)
Same 1.02 -887773 macOS Monterey 12.6 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

@adamsitnik
Copy link
Member

Thank you for the confirmation @dakersnar ! net7regressions--;

@dakersnar dakersnar mentioned this issue Dec 5, 2022
Closed
9 tasks
@jeffhandley jeffhandley added os-linux Linux OS (any supported distro) arch-x64 and removed refs/heads/main labels Dec 28, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Jan 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jeffhandley jeffhandley

@adamsitnik adamsitnik

@kotlarmilos kotlarmilos

Labels
arch-x64 area-System.Memory os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Microbenchmarks] TFMs cleanup + Microsoft.Extensions* update adamsitnik/performance
6 participants
@jeffhandley @adamsitnik @jeffschwMSFT @kotlarmilos @jozkee @dakersnar