Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 141 Improvements on 12/18/2022 12:43:31 PM #11120

Closed
performanceautofiler bot opened this issue Dec 27, 2022 · 6 comments
Closed

[Perf] Linux/x64: 141 Improvements on 12/18/2022 12:43:31 PM #11120

performanceautofiler bot opened this issue Dec 27, 2022 · 6 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesWord - Duration of single invocation 36.08 μs 7.49 μs 0.21 0.07 False
MatchWord - Duration of single invocation 2.24 μs 389.82 ns 0.17 0.07 False
ReplaceWords - Duration of single invocation 121.45 μs 17.58 μs 0.14 0.03 False
Backtracking - Duration of single invocation 624.72 ns 369.05 ns 0.59 0.04 False
SplitWords - Duration of single invocation 254.70 μs 113.95 μs 0.45 0.02 False
ReplaceWords - Duration of single invocation 70.65 μs 12.42 μs 0.18 0.10 False
OneNodeBacktracking - Duration of single invocation 2.03 μs 1.70 μs 0.84 0.03 False
MatchesWord - Duration of single invocation 35.41 μs 7.11 μs 0.20 0.06 False
MatchWord - Duration of single invocation 3.70 μs 533.37 ns 0.14 0.02 False
OneNodeBacktracking - Duration of single invocation 1.36 μs 271.30 ns 0.20 0.05 False
SplitWords - Duration of single invocation 70.37 μs 12.05 μs 0.17 0.13 False
Backtracking - Duration of single invocation 948.42 ns 383.11 ns 0.40 0.03 False
MatchesWords - Duration of single invocation 68.18 μs 10.25 μs 0.15 0.08 False
ReplaceWords - Duration of single invocation 258.78 μs 113.45 μs 0.44 0.02 False
MatchesWords - Duration of single invocation 119.37 μs 15.33 μs 0.13 0.03 False
MatchesWord - Duration of single invocation 62.49 μs 7.93 μs 0.13 0.10 False
SplitWords - Duration of single invocation 121.53 μs 17.00 μs 0.14 0.02 False
Backtracking - Duration of single invocation 2.66 μs 2.38 μs 0.89 0.01 False
OneNodeBacktracking - Duration of single invocation 690.50 ns 253.03 ns 0.37 0.03 False
MatchWord - Duration of single invocation 6.84 μs 3.09 μs 0.45 0.01 False
MatchesWords - Duration of single invocation 255.89 μs 117.06 μs 0.46 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.4914994698660715 < 34.134923858574794.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 323.14790820794354 (T) = (0 -7588.838816984584) / Math.Sqrt((862575.817051592 / (299)) + (165266.72990855563 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.790929776437225 = (36298.037509420734 - 7588.838816984584) / 36298.037509420734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 389.8231096207667 < 2.1324269399415114.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 454.9577952420192 (T) = (0 -389.8934789354446) / Math.Sqrt((4514.776355916569 / (299)) + (19.76819442986105 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8221697701209937 = (2192.5039359209277 - 389.8934789354446) / 2192.5039359209277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.57575727228036 < 115.50642455252859.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 664.79974503212 (T) = (0 -17564.407622785668) / Math.Sqrt((7279859.5814475985 / (299)) + (11189.211263508401 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8560497053993933 = (122017.1703817523 - 17564.407622785668) / 122017.1703817523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 369.04906063523765 < 593.6533121329092.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 321.4331190753153 (T) = (0 -368.3482762244578) / Math.Sqrt((143.8627300641115 / (299)) + (6.084317123923363 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4158500272098044 = (630.5714172424595 - 368.3482762244578) / 630.5714172424595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.94668925120774 < 242.73622346631382.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 411.6728663678483 (T) = (0 -114814.39286913526) / Math.Sqrt((25187555.68858278 / (282)) + (977683.4539012865 / (33))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (282) + (33) - 2, .975) and 0.5528919677313768 = (256793.40245033795 - 114814.39286913526) / 256793.40245033795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.421587283871883 < 67.11468231135066.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 380.7917444846839 (T) = (0 -12556.876759341161) / Math.Sqrt((6330989.540350357 / (299)) + (30025.954449869878 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8183974610350875 = (69144.83041323161 - 12556.876759341161) / 69144.83041323161 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6981974896761578 < 1.9203875762223916.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 137.66014122767538 (T) = (0 -1697.7743596331943) / Math.Sqrt((1419.5763375245722 / (299)) + (28.90929564509956 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1612734985533086 = (2024.228823942918 - 1697.7743596331943) / 2024.228823942918 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.111336962594697 < 33.636852982656805.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 574.3156522435836 (T) = (0 -7128.803676121713) / Math.Sqrt((738516.7309338709 / (299)) + (993.8033443441914 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.801121355763791 = (35844.99332997664 - 7128.803676121713) / 35844.99332997664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 533.3743871474677 < 3.5175438330559463.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 715.060669951745 (T) = (0 -530.9406789932791) / Math.Sqrt((5820.641355477921 / (299)) + (17.019369353739656 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.85755779089807 = (3727.4111539041382 - 530.9406789932791) / 3727.4111539041382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 271.3044746859844 < 1.2880728646253456.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 528.521284284486 (T) = (0 -272.2277152935718) / Math.Sqrt((1098.5998797388243 / (299)) + (11.968352985706344 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7959523434406693 = (1334.137916033437 - 272.2277152935718) / 1334.137916033437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.045294838978702 < 67.03409183365844.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 371.526777794764 (T) = (0 -12238.70548188482) / Math.Sqrt((6470345.695028366 / (299)) + (40335.80607968716 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.8212321778482392 = (68461.45651142437 - 12238.70548188482) / 68461.45651142437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 383.10830651055886 < 901.412874890448.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 484.2003396946704 (T) = (0 -382.5945004969561) / Math.Sqrt((434.9378160199789 / (299)) + (1.187359815435157 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6070955755078546 = (973.7597152067823 - 382.5945004969561) / 973.7597152067823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.25198492968921 < 64.9610060293491.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 384.35023856000845 (T) = (0 -10332.925354449704) / Math.Sqrt((6057599.0330866845 / (299)) + (45852.50163199758 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8455113205804965 = (66884.67655543452 - 10332.925354449704) / 66884.67655543452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.45165515873015 < 244.204683777228.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 423.57788118159385 (T) = (0 -115532.42654192702) / Math.Sqrt((9050101.72483086 / (299)) + (2661719.8672148893 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5497697703655005 = (256607.43978856583 - 115532.42654192702) / 256607.43978856583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.329304496997917 < 113.23762394755853.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 700.9937382302921 (T) = (0 -15377.673766366537) / Math.Sqrt((6459916.4473476745 / (299)) + (16653.317077082556 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8714357350834891 = (119610.79368635404 - 15377.673766366537) / 119610.79368635404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.931600180654158 < 59.521399215624996.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 358.5638716008966 (T) = (0 -7790.256983573591) / Math.Sqrt((6090125.705321033 / (289)) + (20145.418820306684 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.8714267080965036 = (60590.0095442897 - 7790.256983573591) / 60590.0095442897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.996911073702705 < 115.43483147425249.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 683.9204320073237 (T) = (0 -17115.790232166197) / Math.Sqrt((6656341.857929843 / (288)) + (15303.336519679402 / (33))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (288) + (33) - 2, .975) and 0.859854000903676 = (122128.28295156911 - 17115.790232166197) / 122128.28295156911 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.383229634418902 < 2.530595050074358.
IsChangePoint: Marked as a change because one of 12/5/2022 8:20:45 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 62.17090753281283 (T) = (0 -2386.451152242458) / Math.Sqrt((1796.2257283282856 / (298)) + (205.11321068637858 / (33))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (298) + (33) - 2, .975) and 0.08354006596864177 = (2603.9885254392384 - 2386.451152242458) / 2603.9885254392384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.03484650161724 < 657.3133430418428.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 314.45985869812404 (T) = (0 -254.31319705740498) / Math.Sqrt((174.26745485342252 / (299)) + (48.00425705402154 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6383370644250789 = (703.177384359648 - 254.31319705740498) / 703.177384359648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0938318366993602 < 6.485755413087226.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 819.3238369891567 (T) = (0 -3100.8388069101075) / Math.Sqrt((4226.659367691249 / (299)) + (214.6464843807695 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5455427205194489 = (6823.168968608875 - 3100.8388069101075) / 6823.168968608875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.05983085175089 < 242.13728502868184.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 308.225604053091 (T) = (0 -113486.89772535418) / Math.Sqrt((18068397.281722024 / (289)) + (4874718.54924198 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.5546285135715219 = (254814.0174743292 - 113486.89772535418) / 254814.0174743292 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 3.36 μs 388.63 ns 0.12 0.09 False
LastIndexOf_Word_NotFound - Duration of single invocation 1.21 μs 1.05 μs 0.87 0.06 False
IndexOf_Word_NotFound - Duration of single invocation 1.61 μs 234.51 ns 0.15 0.08 False
LastIndexOf_Word_NotFound - Duration of single invocation 1.42 μs 230.05 ns 0.16 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 388.6335807238261 < 3.1872003443668255.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 331.120344835703 (T) = (0 -384.0189838209742) / Math.Sqrt((22589.25866242762 / (299)) + (55.07234078201694 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8834081152460818 = (3293.702513099382 - 384.0189838209742) / 3293.702513099382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0470266361883749 < 1.1477196560290905.
IsChangePoint: Marked as a change because one of 10/27/2022 9:35:54 AM, 11/1/2022 2:33:16 AM, 11/8/2022 11:25:18 AM, 11/10/2022 11:11:41 PM, 12/10/2022 11:25:25 PM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 32.39743603857915 (T) = (0 -1046.8635413246207) / Math.Sqrt((3683.631422022154 / (299)) + (1.3770139310685867 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.09825425718960787 = (1160.9298404469896 - 1046.8635413246207) / 1160.9298404469896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 234.50572586752529 < 1.5397788602384967.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 413.95974076532127 (T) = (0 -233.26174638507945) / Math.Sqrt((3459.366868430624 / (299)) + (0.6563700805561798 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8579863488116164 = (1642.530449947053 - 233.26174638507945) / 1642.530449947053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.05433045893736 < 1.3469426210582482.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 336.8821107980936 (T) = (0 -229.6346621789563) / Math.Sqrt((4194.827235901139 / (299)) + (1.2015120324640476 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8462024205351516 = (1493.0967247858466 - 229.6346621789563) / 1493.0967247858466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 675.98 ns 390.05 ns 0.58 0.08 False
TwoArguments_DefineMessage - Duration of single invocation 536.29 ns 256.55 ns 0.48 0.04 False
FourArguments_DefineMessage - Duration of single invocation 1.05 μs 450.85 ns 0.43 0.02 False
FourArguments_EnumerableArgument - Duration of single invocation 3.58 μs 2.91 μs 0.81 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 390.0496035572363 < 644.4920669932276.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 172.1873783409561 (T) = (0 -392.99658134529176) / Math.Sqrt((189.70731935738422 / (299)) + (72.3488200964479 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4241807511285213 = (682.4999027307744 - 392.99658134529176) / 682.4999027307744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 256.54939985490995 < 510.4968817250954.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 257.60152374972324 (T) = (0 -258.6119089050357) / Math.Sqrt((106.45001145642051 / (299)) + (26.70242869747459 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5181233473224988 = (536.6765695496629 - 258.6119089050357) / 536.6765695496629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 450.84791166950833 < 0.9991890235997299.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 457.2221769295112 (T) = (0 -452.9850555547874) / Math.Sqrt((298.535662735949 / (299)) + (21.510232957292914 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5645823198132545 = (1040.346031333655 - 452.9850555547874) / 1040.346031333655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_EnumerableArgument

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9056516887218735 < 3.403747550548424.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 42.48876522658255 (T) = (0 -2930.736947802502) / Math.Sqrt((14211.923360637933 / (299)) + (4006.33250517245 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1585560949106894 = (3482.985532459748 - 2930.736947802502) / 3482.985532459748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfShort - Duration of single invocation 488.49 ns 67.69 ns 0.14 0.07 False
IndexOfChar - Duration of single invocation 493.17 ns 67.77 ns 0.14 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.IndexOfShort

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.6889732818223 < 459.9549083829024.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 414.1706678961022 (T) = (0 -68.02358987666484) / Math.Sqrt((315.79749955464814 / (299)) + (0.1973314761276032 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8625431627392224 = (494.87236307942413 - 68.02358987666484) / 494.87236307942413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Array.IndexOfChar

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.77221242205448 < 469.244941310301.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 429.5505126131521 (T) = (0 -68.53582934149819) / Math.Sqrt((290.50035297372426 / (299)) + (0.4673907397948008 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8615471020328083 = (495.0118801972542 - 68.53582934149819) / 495.0118801972542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToHexString - Duration of single invocation 309.58 ns 270.81 ns 0.87 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToHexString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 270.8064687824835 < 291.8194594569812.
IsChangePoint: Marked as a change because one of 10/18/2022 9:26:37 AM, 11/8/2022 11:25:18 AM, 11/20/2022 12:28:28 PM, 12/2/2022 5:47:06 PM, 12/7/2022 5:17:51 AM, 12/14/2022 12:26:48 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.505299486483095 (T) = (0 -264.66809493701743) / Math.Sqrt((563.9940468214305 / (299)) + (46.304055031091224 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.0611531312423318 = (281.9076291826379 - 264.66809493701743) / 281.9076291826379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BuilderToString - Duration of single invocation 540.92 ns 484.96 ns 0.90 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.BuilderToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 484.9555276603496 < 515.4646697186745.
IsChangePoint: Marked as a change because one of 11/30/2022 7:06:59 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 28.89099578296909 (T) = (0 -477.8865906083596) / Math.Sqrt((219.32180289038146 / (299)) + (224.8564431867827 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14243084329052036 = (557.2572041210365 - 477.8865906083596) / 557.2572041210365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 989.27 ns 830.89 ns 0.84 0.03 False
TryParseGeneric - Duration of single invocation 949.41 ns 787.10 ns 0.83 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 830.8933694067352 < 942.9167048767055.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 131.3374444117018 (T) = (0 -831.8687559603646) / Math.Sqrt((722.8470934539345 / (299)) + (13.291920317530002 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2095769963926477 = (1052.4349015196435 - 831.8687559603646) / 1052.4349015196435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red, Orange, Yellow, Green, Blue")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 787.098328299354 < 901.7542878073702.
IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/9/2022 1:52:32 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 86.94531296552927 (T) = (0 -791.8562541516728) / Math.Sqrt((594.3533343601202 / (299)) + (22.98143532208085 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.15246369904751364 = (934.3036437044187 - 791.8562541516728) / 934.3036437044187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 79.69 ns 19.58 ns 0.25 0.01 False
IndexOfValue - Duration of single invocation 873.81 ns 198.38 ns 0.23 0.00 False
LastIndexOfValue - Duration of single invocation 28.11 ns 21.23 ns 0.76 0.13 False
LastIndexOfValue - Duration of single invocation 252.24 ns 190.64 ns 0.76 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfValue(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.58282073955806 < 75.58864128286973.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 1857.0929912624435 (T) = (0 -19.60690114724334) / Math.Sqrt((0.05484881509038702 / (70)) + (0.0087325112117703 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.7540877867583673 = (79.7313028449614 - 19.60690114724334) / 79.7313028449614 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 198.38260248411706 < 830.5029549987894.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 949.9071690875126 (T) = (0 -202.9409936973003) / Math.Sqrt((1.2178753812641694 / (299)) + (16.29481232116313 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7675852662360726 = (873.1847177272131 - 202.9409936973003) / 873.1847177272131 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.227440909782786 < 26.7244998603779.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 5.012255216671485 (T) = (0 -21.511728386927185) / Math.Sqrt((8.849186960582095 / (70)) + (0.18080706360656484 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.07801820609966942 = (23.332053332554935 - 21.511728386927185) / 23.332053332554935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 190.6396286854151 < 239.52135033961713.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 28.31490766911281 (T) = (0 -191.6015682757201) / Math.Sqrt((141.02228549388087 / (299)) + (0.8494055748150168 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09438568827855907 = (211.57082633943077 - 191.6015682757201) / 211.57082633943077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 31.20 μs 24.25 μs 0.78 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.List(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.246288100775192 < 29.648541391480137.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 61.67955320237645 (T) = (0 -23886.816270110423) / Math.Sqrt((18390941.8669281 / (299)) + (78032.63307130575 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3948895017057748 = (39475.13113298498 - 23886.816270110423) / 39475.13113298498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 2.17 μs 553.62 ns 0.25 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 553.6171811694757 < 2.0569059143051955.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9004.074898820712 (T) = (0 -552.3972711981678) / Math.Sqrt((6.8947940875619285 / (299)) + (0.2990935841191458 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7449909098515904 = (2166.186589178782 - 552.3972711981678) / 2166.186589178782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseUInt64 - Duration of single invocation 51.80 ns 41.39 ns 0.80 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.39417899185678 < 49.386159856100676.
IsChangePoint: Marked as a change because one of 10/18/2022 9:26:37 AM, 10/22/2022 11:45:45 AM, 11/2/2022 9:35:40 AM, 11/10/2022 11:11:41 PM, 11/29/2022 11:48:02 PM, 12/7/2022 5:17:51 AM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 17.383528857435905 (T) = (0 -41.47006241489047) / Math.Sqrt((28.863639540885707 / (299)) + (0.05305904195056147 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11607663072614975 = (46.9159022789028 - 41.47006241489047) / 46.9159022789028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TwoArguments - Duration of single invocation 537.68 ns 257.22 ns 0.48 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.Formatting*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.Formatting.TwoArguments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 257.2206729885301 < 512.094453871574.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 433.7478148038909 (T) = (0 -256.90567783427457) / Math.Sqrt((82.2401073165214 / (299)) + (4.627127692664648 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5210717392818505 = (536.4178707037382 - 256.90567783427457) / 536.4178707037382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadAllLines - Duration of single invocation 146.59 μs 69.92 μs 0.48 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_File.ReadAllLines

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.9164598931624 < 139.01694219624096.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.8798120368542 (T) = (0 -69874.46626249814) / Math.Sqrt((1245305948.0360103 / (299)) + (637138.0341380979 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.41144187981120045 = (118721.43780818721 - 69874.46626249814) / 118721.43780818721 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 278.04 μs 42.52 μs 0.15 0.08 False
ReadLine - Duration of single invocation 589.41 μs 148.05 μs 0.25 0.09 False
ReadLine - Duration of single invocation 3.01 ms 283.63 μs 0.09 0.12 False
ReadLine - Duration of single invocation 258.54 μs 36.00 μs 0.14 0.10 False
ReadLine - Duration of single invocation 263.14 μs 36.63 μs 0.14 0.10 False
ReadLine - Duration of single invocation 333.61 μs 58.77 μs 0.18 0.08 False
ReadLine - Duration of single invocation 257.35 μs 27.40 μs 0.11 0.08 False
ReadLine - Duration of single invocation 1.49 ms 253.01 μs 0.17 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.5199465990991 < 263.85736171559347.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 320.23440825425297 (T) = (0 -43664.954506209564) / Math.Sqrt((135967429.39970064 / (299)) + (1239453.0655105016 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8372862765070735 = (268354.466782931 - 43664.954506209564) / 268354.466782931 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 148.0537931290064 < 557.4286235265313.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 379.54439801212385 (T) = (0 -147691.95948916004) / Math.Sqrt((371705785.2111161 / (299)) + (1150008.1208751837 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7439299165036068 = (576763.8197815496 - 147691.95948916004) / 576763.8197815496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 283.6346712121212 < 2.865346724036458.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 402.4823197113765 (T) = (0 -283759.6181155403) / Math.Sqrt((13069285445.045752 / (299)) + (6189355.6518301545 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.9038238307172861 = (2950415.0584477633 - 283759.6181155403) / 2950415.0584477633 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.99679568725426 < 245.77337233618752.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 310.3804756849083 (T) = (0 -36920.420158816596) / Math.Sqrt((126147992.8899433 / (299)) + (1602492.157985814 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8522063863844783 = (249810.65998469578 - 36920.420158816596) / 249810.65998469578 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.626025221128174 < 249.99974399403573.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 306.96473138375836 (T) = (0 -37683.61894513826) / Math.Sqrt((133292155.58440977 / (299)) + (1643367.709243371 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8515116174251229 = (253781.59753431115 - 37683.61894513826) / 253781.59753431115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 58.773987323427505 < 315.7358442863475.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 337.7779440264945 (T) = (0 -59123.69367549339) / Math.Sqrt((167090041.42548525 / (299)) + (1657140.5137981484 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8168015503033145 = (322730.31662321475 - 59123.69367549339) / 322730.31662321475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.39937463578088 < 244.54349355044886.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 299.321841754883 (T) = (0 -28352.24774229751) / Math.Sqrt((143704568.8597055 / (299)) + (1886017.8715197493 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.885608712961961 = (247853.21047108612 - 28352.24774229751) / 247853.21047108612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.0081609457672 < 1.4155912535917208.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 358.6244216533816 (T) = (0 -253213.28599541495) / Math.Sqrt((3366124819.597776 / (299)) + (1841363.5863326744 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8265043311667128 = (1459479.0042783648 - 253213.28599541495) / 1459479.0042783648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 53.77 μs 26.89 μs 0.50 0.05 False
Utf8JsonReaderCommentParsing - Duration of single invocation 157.50 μs 43.53 μs 0.28 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 18.13 μs 4.93 μs 0.27 0.02 False
Utf8JsonReaderCommentParsing - Duration of single invocation 14.99 μs 3.39 μs 0.23 0.05 False
Utf8JsonReaderCommentParsing - Duration of single invocation 157.43 μs 43.64 μs 0.28 0.08 False
Utf8JsonReaderCommentParsing - Duration of single invocation 53.96 μs 26.86 μs 0.50 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 14.94 μs 3.31 μs 0.22 0.08 False
Utf8JsonReaderCommentParsing - Duration of single invocation 18.05 μs 4.86 μs 0.27 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.889114736745213 < 51.115409786214286.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 175.36294884510505 (T) = (0 -27154.324587019124) / Math.Sqrt((5880420.942926452 / (267)) + (16067.312462031785 / (33))) is greater than 1.9679565064968887 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (267) + (33) - 2, .975) and 0.49211177840972814 = (53465.15912890239 - 27154.324587019124) / 53465.15912890239 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.52795543196329 < 149.77088678408873.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 542.8046239597594 (T) = (0 -43742.13675751233) / Math.Sqrt((7064505.62185995 / (289)) + (641655.694731248 / (33))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (33) - 2, .975) and 0.7221976889405783 = (157457.78568471278 - 43742.13675751233) / 157457.78568471278 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.928141168564199 < 17.23726870463905.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 892.3753280138599 (T) = (0 -4907.1507310490015) / Math.Sqrt((47579.13105095942 / (285)) + (1834.914076683891 / (33))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (285) + (33) - 2, .975) and 0.7306669371641571 = (18219.637349313787 - 4907.1507310490015) / 18219.637349313787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.389298417498373 < 14.217211075757577.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 736.5622048067122 (T) = (0 -3323.840871762184) / Math.Sqrt((50666.77640625798 / (299)) + (2760.917779478671 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7790340002358815 = (15042.318163474876 - 3323.840871762184) / 15042.318163474876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.64141455602708 < 149.55372883268012.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 640.5364269333126 (T) = (0 -43532.476292036314) / Math.Sqrt((9226806.501219202 / (299)) + (20256.025879259934 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7230179648704337 = (157167.14721831234 - 43532.476292036314) / 157167.14721831234 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongMultiLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.858331697726666 < 51.1202571760551.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 204.63941295712223 (T) = (0 -27155.527765655293) / Math.Sqrt((4855705.314318245 / (299)) + (20229.384665412832 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4945128362823341 = (53721.49821953284 - 27155.527765655293) / 53721.49821953284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.307326380059335 < 14.219883991313305.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 757.927424248937 (T) = (0 -3284.6745379414747) / Math.Sqrt((63722.53408862642 / (299)) + (917.7751720378384 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.7820441779057392 = (15070.368418609762 - 3284.6745379414747) / 15070.368418609762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongSingleLine)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.859627000962734 < 17.14820491512653.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 898.8839743543975 (T) = (0 -4857.334850854114) / Math.Sqrt((60594.444363240815 / (299)) + (547.4340789200832 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7326296969484276 = (18167.069399316173 - 4857.334850854114) / 18167.069399316173 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 130.06 μs 100.36 μs 0.77 0.15 False
List - Duration of single invocation 444.12 μs 105.74 μs 0.24 0.00 False
Array - Duration of single invocation 445.59 μs 110.84 μs 0.25 0.00 False
ImmutableArray - Duration of single invocation 447.62 μs 113.35 μs 0.25 0.00 False
ICollection - Duration of single invocation 447.51 μs 109.77 μs 0.25 0.00 False
Queue - Duration of single invocation 443.84 μs 108.27 μs 0.24 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.Stack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 100.36265407509156 < 123.41227847187376.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 33.375895993058 (T) = (0 -100109.69950763774) / Math.Sqrt((33069961.809593566 / (299)) + (35489.601836576825 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10024517001589774 = (111263.30881647678 - 100109.69950763774) / 111263.30881647678 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 105.7368760702055 < 422.26673328707835.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 1568.6958523671747 (T) = (0 -107082.94429761976) / Math.Sqrt((234332.60657476744 / (299)) + (1500949.9544828183 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.759096703933785 = (444505.9326551795 - 107082.94429761976) / 444505.9326551795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 110.8424770355974 < 423.9529821312957.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 4290.0944615630715 (T) = (0 -110624.6143150237) / Math.Sqrt((295198.5252750961 / (299)) + (169269.98908080754 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7520458566039967 = (446149.48877199047 - 110624.6143150237) / 446149.48877199047 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.35154835562989 < 425.35928387436223.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 1797.489142048073 (T) = (0 -113041.85449893004) / Math.Sqrt((262241.0734289383 / (299)) + (1118404.7036775697 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.747790089202385 = (448205.4418140613 - 113041.85449893004) / 448205.4418140613 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 109.77197421328673 < 422.59385180398436.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 2247.493966096527 (T) = (0 -108737.67162456783) / Math.Sqrt((260393.9063823664 / (299)) + (708755.4333712102 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7554937648233023 = (444723.51204453426 - 108737.67162456783) / 444723.51204453426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.27483417487683 < 421.87306778604494.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 1540.8911897404203 (T) = (0 -107544.17535650764) / Math.Sqrt((269630.0545420913 / (299)) + (1543924.8190466543 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7578026696492012 = (444035.3458922961 - 107544.17535650764) / 444035.3458922961 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Format_OneArg - Duration of single invocation 3.63 μs 2.59 μs 0.71 0.02 False
Replace_Char - Duration of single invocation 14.88 ns 10.25 ns 0.69 0.18 False
ToLowerInvariant - Duration of single invocation 379.02 ns 354.88 ns 0.94 0.00 False
Format_OneArg - Duration of single invocation 2.44 μs 1.14 μs 0.47 0.02 False
IndexOfAny - Duration of single invocation 1.11 μs 113.74 ns 0.10 0.11 False
Format_MultipleArgs - Duration of single invocation 1.58 μs 568.74 ns 0.36 0.04 False
Replace_String - Duration of single invocation 308.15 ns 176.92 ns 0.57 0.04 False
Replace_Char - Duration of single invocation 145.42 ns 27.33 ns 0.19 0.08 False
Replace_String - Duration of single invocation 163.08 ns 47.71 ns 0.29 0.08 False
Replace_String - Duration of single invocation 650.10 ns 214.17 ns 0.33 0.09 False
ToUpperInvariant - Duration of single invocation 378.19 ns 354.09 ns 0.94 0.00 False
ToUpper - Duration of single invocation 385.83 ns 362.29 ns 0.94 0.00 False
ToLower - Duration of single invocation 384.98 ns 358.58 ns 0.93 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5872275937551628 < 3.4653450808948.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 158.458256491921 (T) = (0 -2603.3364593462024) / Math.Sqrt((7153.363207914268 / (299)) + (528.8194987245336 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.27782931268591254 = (3604.8769426360777 - 2603.3364593462024) / 3604.8769426360777 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.25082859877133 < 14.178267849748872.
IsChangePoint: Marked as a change because one of 11/11/2022 10:59:41 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 50.27631535079896 (T) = (0 -10.01838291839502) / Math.Sqrt((2.5938528378943486 / (299)) + (0.12511013808197413 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3591051042497444 = (15.631865669123679 - 10.01838291839502) / 15.631865669123679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 354.87895040631884 < 359.639390319056.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 110.0580016435118 (T) = (0 -355.5473148527429) / Math.Sqrt((1.1170161662904843 / (299)) + (0.6586705478812802 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.05852479983210542 = (377.6491561215183 - 355.5473148527429) / 377.6491561215183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:D5}, {0:E} - {0:F4}{0:G}{0:N}  {0:X} !!", o: 8)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1403323449740101 < 2.319331255792199.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 482.7112714914453 (T) = (0 -1143.4981961832755) / Math.Sqrt((1798.4373212768353 / (299)) + (36.12622953388534 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5295386818221491 = (2430.589194053555 - 1143.4981961832755) / 2430.589194053555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.IndexOfAny

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 113.73556904840054 < 1.0520645282344727.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 308.7036073762488 (T) = (0 -116.52075842601856) / Math.Sqrt((2647.7706400861634 / (299)) + (20.37001234650253 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.890759131727914 = (1066.6407203556873 - 116.52075842601856) / 1066.6407203556873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_MultipleArgs

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 568.7384623453862 < 1.4961765850577822.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 471.3323227633653 (T) = (0 -571.9798298729085) / Math.Sqrt((1149.2871776395477 / (299)) + (17.661371286574326 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6329435625011066 = (1558.2885121709187 - 571.9798298729085) / 1558.2885121709187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 176.9200977983937 < 294.7274669963463.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 327.85068921077317 (T) = (0 -177.18980297032087) / Math.Sqrt((44.17757598850232 / (299)) + (0.7502145130854921 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4330991870281091 = (312.5587385232884 - 177.18980297032087) / 312.5587385232884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.329158720478542 < 136.9041117378089.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 447.08044921100276 (T) = (0 -27.039933043031024) / Math.Sqrt((21.510297981165586 / (299)) + (0.08694350022312154 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.818683257406158 = (149.13092225356016 - 27.039933043031024) / 149.13092225356016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.7063900832487 < 154.7756252654364.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 227.48913654310036 (T) = (0 -48.41230176342438) / Math.Sqrt((19.207292504146317 / (299)) + (6.391360139589687 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7047015631841015 = (163.94364387918063 - 48.41230176342438) / 163.94364387918063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 214.16986528726733 < 615.0861226449323.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 439.8052320603239 (T) = (0 -213.29044534715965) / Math.Sqrt((281.9126946210674 / (299)) + (1.3197215697288718 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6715104157386389 = (649.3065703339203 - 213.29044534715965) / 649.3065703339203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 354.09099021104356 < 359.232649856662.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 118.8686829275623 (T) = (0 -354.6137669241216) / Math.Sqrt((1.7413731329889688 / (299)) + (0.5930818929051991 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.061918047785121826 = (378.0200291529469 - 354.6137669241216) / 378.0200291529469 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 362.29113200032555 < 366.51528571147435.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 69.4230076519876 (T) = (0 -362.96521640162234) / Math.Sqrt((1.89123614972717 / (299)) + (1.6361631694956023 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.057720306913869666 = (385.1990221850669 - 362.96521640162234) / 385.1990221850669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 358.58463813176 < 366.2229174302378.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 152.80725471489035 (T) = (0 -358.20228076614995) / Math.Sqrt((2.097950699795985 / (299)) + (0.4351878674450094 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.07006504047598316 = (385.1906814531354 - 358.20228076614995) / 385.1906814531354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 345.95 ns 95.77 ns 0.28 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 95.77408990723346 < 328.6481247386523.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 2063.7555611886155 (T) = (0 -97.51392164303405) / Math.Sqrt((1.1655835093317475 / (299)) + (0.35048281674500054 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.718318836254193 = (346.18545431398456 - 97.51392164303405) / 346.18545431398456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SetEnvironmentVariable - Duration of single invocation 333.82 ns 166.60 ns 0.50 0.04 False
GetEnvironmentVariable - Duration of single invocation 285.90 ns 118.41 ns 0.41 0.05 False
ExpandEnvironmentVariables - Duration of single invocation 754.84 ns 347.52 ns 0.46 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Environment.SetEnvironmentVariable

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 166.5995185000856 < 316.7228534604901.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 350.36554205217163 (T) = (0 -167.85942205883924) / Math.Sqrt((56.947432351245645 / (299)) + (2.1516007646276845 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5134707592894465 = (345.01404646036957 - 167.85942205883924) / 345.01404646036957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.GetEnvironmentVariable

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 118.4106528065814 < 271.2246850087415.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 263.561417404169 (T) = (0 -120.23320811676638) / Math.Sqrt((44.57265841916926 / (299)) + (8.935063889563212 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.586836483667981 = (291.00635308793034 - 120.23320811676638) / 291.00635308793034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.ExpandEnvironmentVariables

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 347.51529117382813 < 708.0550116100057.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 348.8916832263832 (T) = (0 -345.686694041538) / Math.Sqrt((400.9811212810837 / (299)) + (2.196321755462181 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5449240467504575 = (759.6241716863018 - 345.686694041538) / 759.6241716863018 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddChainedConfigurationEmpty - Duration of single invocation 140.29 ms 45.29 ms 0.32 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.2943971547619 < 133.26034916380493.
IsChangePoint: Marked as a change because one of 11/8/2022 3:14:34 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 51.51731045869711 (T) = (0 -45781785.19228655) / Math.Sqrt((663884155050755.4 / (299)) + (340922471105.4249 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6269570912025394 = (122725252.5449914 - 45781785.19228655) / 122725252.5449914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteFormat - Duration of single invocation 672.97 ns 312.30 ns 0.46 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_StreamWriter.WriteFormat

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 312.30140279390764 < 640.3784805461262.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 263.17034572322547 (T) = (0 -308.32406918103015) / Math.Sqrt((397.53764670008815 / (299)) + (17.425463382541725 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5377513288425284 = (667.0090979580019 - 308.32406918103015) / 667.0090979580019 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayLeftShift - Duration of single invocation 327.33 ns 294.85 ns 0.90 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 294.8492528939928 < 310.3726829208488.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/15/2022 2:06:03 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.125330405662485 (T) = (0 -297.52705106114917) / Math.Sqrt((93.39090292863325 / (299)) + (118.7815535506977 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.08583858360114345 = (325.4644592562147 - 297.52705106114917) / 325.4644592562147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 35.06 ns 31.79 ns 0.91 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryParse(value: "12345")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.793170505088035 < 33.313150601461544.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 14.560811876253995 (T) = (0 -32.3398659865626) / Math.Sqrt((1.7238199433737584 / (299)) + (0.34594036878324563 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.05427697602234407 = (34.19591695097261 - 32.3398659865626) / 34.19591695097261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NestedScopes_TwoMessages - Duration of single invocation 2.23 μs 1.98 μs 0.89 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9831780207671115 < 2.095706338547357.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 36.79735398184112 (T) = (0 -1865.115950600329) / Math.Sqrt((3913.50350443891 / (299)) + (2196.255556953294 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.14970931291250159 = (2193.5039145129444 - 1865.115950600329) / 2193.5039145129444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HtmlDecode_Entities - Duration of single invocation 1.60 μs 784.91 ns 0.49 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 784.9085864987201 < 1.5252690852543969.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 198.30755285310582 (T) = (0 -786.8368332801429) / Math.Sqrt((1074.9699767600243 / (299)) + (409.8298642941109 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5051278386126073 = (1589.9799881129227 - 786.8368332801429) / 1589.9799881129227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 1.77 ms 529.91 μs 0.30 0.05 False
ReadLine - Duration of single invocation 783.73 μs 315.82 μs 0.40 0.03 False
ReadLine - Duration of single invocation 404.81 μs 178.41 μs 0.44 0.05 False
ReadLine - Duration of single invocation 419.68 μs 179.06 μs 0.43 0.05 False
ReadLine - Duration of single invocation 483.20 μs 201.36 μs 0.42 0.04 False
ReadLine - Duration of single invocation 412.96 μs 181.69 μs 0.44 0.05 False
ReadLine - Duration of single invocation 400.53 μs 174.46 μs 0.44 0.04 False
ReadLine - Duration of single invocation 3.46 ms 739.52 μs 0.21 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 529.9146751847292 < 1.6798028683217594.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.62009894638844 (T) = (0 -529770.4350855268) / Math.Sqrt((410871019245.57434 / (299)) + (3398765.8300698786 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.5906505837693593 = (1294176.5984760488 - 529770.4350855268) / 1294176.5984760488 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 315.81669880952376 < 743.9108386927082.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 21.207025982247618 (T) = (0 -316887.8542201894) / Math.Sqrt((56370801305.7756 / (299)) + (6866452.511531665 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4790042633472701 = (608235.0236800713 - 316887.8542201894) / 608235.0236800713 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 178.40795175527597 < 385.0396370673077.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.42715493224627 (T) = (0 -180026.16796479607) / Math.Sqrt((9739855521.452803 / (299)) + (8980232.167913035 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4569117618029223 = (331486.03726429364 - 180026.16796479607) / 331486.03726429364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 179.05640518465913 < 397.89372749255955.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 24.975078852381586 (T) = (0 -181654.78136377686) / Math.Sqrt((11530488848.400528 / (299)) + (9168788.670381188 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.46145490523638105 = (337306.5377997914 - 181654.78136377686) / 337306.5377997914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 201.3600959821428 < 459.02443801784466.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 23.371432124477284 (T) = (0 -203942.45131712966) / Math.Sqrt((17606634360.046265 / (299)) + (6084649.628470349 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4683009455887798 = (383567.45159715664 - 203942.45131712966) / 383567.45159715664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 181.6939740448505 < 393.5017687077702.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.916506369165702 (T) = (0 -184082.5949941215) / Math.Sqrt((9756104100.565699 / (299)) + (9557204.063879425 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.45620649700440447 = (338515.6203229086 - 184082.5949941215) / 338515.6203229086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.45601103219695 < 378.0013363504463.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 26.490679377906705 (T) = (0 -176590.4115912521) / Math.Sqrt((9355070103.158379 / (299)) + (10734090.759908514 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.45753909401267073 = (325535.7384138884 - 176590.4115912521) / 325535.7384138884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 739.5156636904763 < 3.285268612700893.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 20.538867624252674 (T) = (0 -740536.0041688025) / Math.Sqrt((1991870490644.8943 / (299)) + (1187762.5295497833 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6936029956673248 = (2416916.5941477483 - 740536.0041688025) / 2416916.5941477483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 877.75 μs 210.20 μs 0.24 0.00 False
ICollection - Duration of single invocation 876.28 μs 208.00 μs 0.24 0.00 False
Array - Duration of single invocation 877.57 μs 202.10 μs 0.23 0.01 False
List - Duration of single invocation 875.87 μs 208.39 μs 0.24 0.00 False
ImmutableArray - Duration of single invocation 879.27 μs 211.49 μs 0.24 0.00 False
Stack - Duration of single invocation 245.62 μs 189.79 μs 0.77 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.Queue(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 210.1981749358974 < 834.2019492347756.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 1163.2363206039122 (T) = (0 -207607.0121562663) / Math.Sqrt((1003464.5944484484 / (299)) + (10834759.784941018 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7634206080738969 = (877536.3334314156 - 207607.0121562663) / 877536.3334314156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.00179416666666 < 832.0034943142363.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 1139.8571422252794 (T) = (0 -205134.74961222897) / Math.Sqrt((1392406.2534112646 / (299)) + (11254112.384961307 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7656457182289723 = (875318.9745969855 - 205134.74961222897) / 875318.9745969855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 202.09855606303418 < 833.51116328125.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 948.6691653608669 (T) = (0 -205912.23368311662) / Math.Sqrt((9909719.58758095 / (299)) + (15410070.844557332 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.765154731457388 = (876799.583661846 - 205912.23368311662) / 876799.583661846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.39217825 < 830.369296405423.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 7251.202240429154 (T) = (0 -207995.82146500447) / Math.Sqrt((1237649.253047415 / (299)) + (142740.63300072568 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7623276285146848 = (875136.7277784568 - 207995.82146500447) / 875136.7277784568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 211.4943954954955 < 835.1488004778757.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 1194.1442221124037 (T) = (0 -208275.15143873042) / Math.Sqrt((3314637.345695316 / (299)) + (10045710.743891843 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7630598087458068 = (879019.9346774796 - 208275.15143873042) / 879019.9346774796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 189.79425863453812 < 233.732031266276.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 33.3388727839395 (T) = (0 -189938.7123945818) / Math.Sqrt((130273948.81403618 / (299)) + (41558.26630973738 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.10396815859151871 = (211977.63697327458 - 189938.7123945818) / 211977.63697327458 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 7.23 μs 654.97 ns 0.09 0.04 False
LastIndexOfValue - Duration of single invocation 132.52 ns 21.32 ns 0.16 0.10 False
IndexOfAnyTwoValues - Duration of single invocation 417.12 ns 36.41 ns 0.09 0.08 False
IndexOfAnyThreeValues - Duration of single invocation 5.53 μs 445.62 ns 0.08 0.03 False
LastIndexOfValue - Duration of single invocation 20.18 ns 10.90 ns 0.54 0.01 False
IndexOfAnyThreeValues - Duration of single invocation 16.14 ns 14.29 ns 0.89 0.07 False
IndexOfAnyTwoValues - Duration of single invocation 3.91 μs 356.95 ns 0.09 0.11 False
IndexOfAnyFourValues - Duration of single invocation 749.45 ns 61.03 ns 0.08 0.04 False
IndexOfValue - Duration of single invocation 225.65 ns 19.86 ns 0.09 0.09 False
IndexOfAnyThreeValues - Duration of single invocation 568.44 ns 44.21 ns 0.08 0.03 False
LastIndexOfValue - Duration of single invocation 1.59 μs 190.91 ns 0.12 0.07 False
IndexOfValue - Duration of single invocation 12.18 ns 10.56 ns 0.87 0.17 False
IndexOfValue - Duration of single invocation 1.92 μs 207.06 ns 0.11 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 654.9713897397868 < 6.876812812915043.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 591.3990267998396 (T) = (0 -664.4263268116017) / Math.Sqrt((36125.3757874321 / (299)) + (47.3079413623631 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.9077626935279521 = (7203.444595522238 - 664.4263268116017) / 7203.444595522238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.31888612157573 < 126.06724505948111.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 214.5003834234511 (T) = (0 -21.347964782094266) / Math.Sqrt((20.193740308657496 / (70)) + (0.16246450143739644 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.8447828781856597 = (137.5361463513618 - 21.347964782094266) / 137.5361463513618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyTwoValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.407693122088105 < 396.57106155616555.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 196.82341744859104 (T) = (0 -36.473411550722794) / Math.Sqrt((237.53251574296186 / (70)) + (0.23146515946503324 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.90868309820898 = (399.41577994173184 - 36.473411550722794) / 399.41577994173184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 445.62403954943863 < 5.256428179371186.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 925.790966389358 (T) = (0 -482.7652482458344) / Math.Sqrt((5728.560284512684 / (299)) + (357.25220850360137 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.9130505825929774 = (5552.253972973063 - 482.7652482458344) / 5552.253972973063 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.898560445108718 < 19.174598209420292.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 885.5565190352412 (T) = (0 -10.901836873922845) / Math.Sqrt((0.007285508830349987 / (70)) + (0.0001714024351940572 / (32))) is greater than 1.9839715185231144 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (32) - 2, .975) and 0.45938846222289714 = (20.165749548648613 - 10.901836873922845) / 20.165749548648613 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyThreeValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.293761778345145 < 15.211888397716955.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 11.718303913284116 (T) = (0 -14.285995124209215) / Math.Sqrt((1.6931685847548483 / (69)) + (0.028747772285029873 / (33))) is greater than 1.9839715185231144 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (69) + (33) - 2, .975) and 0.11563436271180298 = (16.153946424259015 - 14.285995124209215) / 16.153946424259015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 356.9544282746498 < 3.7043566749752705.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 314.55198290644586 (T) = (0 -372.48012372873984) / Math.Sqrt((31159.47415457746 / (299)) + (415.16115818973316 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.9012478331978084 = (3771.867856579259 - 372.48012372873984) / 3771.867856579259 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFourValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.02630794917973 < 710.702911556126.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 349.64580682339204 (T) = (0 -61.63724059910931) / Math.Sqrt((270.172552496951 / (70)) + (1.6277020750032418 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.9181360259545998 = (752.9226539249904 - 61.63724059910931) / 752.9226539249904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.861165757846006 < 212.43026317601286.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 191.35827030067867 (T) = (0 -19.826714731538317) / Math.Sqrt((82.67576261321229 / (70)) + (0.018450380652983614 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.9129794782289029 = (227.83952943526862 - 19.826714731538317) / 227.83952943526862 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyThreeValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.21189408701242 < 540.695407037935.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 506.52920736512414 (T) = (0 -45.18618953305143) / Math.Sqrt((73.30217787403225 / (70)) + (1.6357076561472788 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.9215041812648793 = (575.6509105985564 - 45.18618953305143) / 575.6509105985564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 190.90509313237595 < 1.5065661081635662.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 344.98490848299195 (T) = (0 -191.27071310531025) / Math.Sqrt((5486.402037991181 / (299)) + (0.823866709674335 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8854701159374903 = (1670.0507004871824 - 191.27071310531025) / 1670.0507004871824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.555121705388357 < 11.886185650136547.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 8.52138690174041 (T) = (0 -10.55778419148018) / Math.Sqrt((4.703427340027172 / (70)) + (0.001354320415315114 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.173061843509115 = (12.767320154003022 - 10.55778419148018) / 12.767320154003022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 207.0562774821158 < 1.8307734960708986.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 372.49274925611076 (T) = (0 -206.5174265117178) / Math.Sqrt((6617.965185929766 / (299)) + (0.4071102405519884 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8946044461402687 = (1959.451029467215 - 206.5174265117178) / 1959.451029467215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 27, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 113.23 ns 18.53 ns 0.16 0.06 False
IndexOfAnyTwoValues - Duration of single invocation 1.41 μs 303.73 ns 0.22 0.07 False
LastIndexOfValue - Duration of single invocation 57.20 ns 19.87 ns 0.35 0.09 False
LastIndexOfAnyValues - Duration of single invocation 94.26 ns 33.27 ns 0.35 0.10 False
IndexOfAnyFourValues - Duration of single invocation 128.20 ns 54.41 ns 0.42 0.38 False
LastIndexOfValue - Duration of single invocation 710.08 ns 174.33 ns 0.25 0.03 False
IndexOfAnyThreeValues - Duration of single invocation 2.04 μs 413.07 ns 0.20 0.07 False
IndexOfValue - Duration of single invocation 12.85 ns 10.19 ns 0.79 0.23 False
LastIndexOfAnyValues - Duration of single invocation 1.26 μs 308.32 ns 0.25 0.07 False
IndexOfAnyFourValues - Duration of single invocation 1.36 μs 507.51 ns 0.37 0.15 False
IndexOfAnyThreeValues - Duration of single invocation 263.82 ns 43.24 ns 0.16 0.05 False
IndexOfValue - Duration of single invocation 800.07 ns 191.36 ns 0.24 0.08 False
IndexOfAnyFourValues - Duration of single invocation 30.57 ns 27.62 ns 0.90 0.04 False
IndexOfAnyTwoValues - Duration of single invocation 186.40 ns 33.83 ns 0.18 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfValue(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.53229050421954 < 108.17086580264137.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 221.3427542079357 (T) = (0 -18.566171374101998) / Math.Sqrt((13.494204625066715 / (70)) + (0.008349417050543097 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.8396881490203101 = (115.81284390792894 - 18.566171374101998) / 115.81284390792894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 303.7257255588367 < 1.335327674762286.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 601.9234262390617 (T) = (0 -306.4294378530592) / Math.Sqrt((939.9734237256469 / (299)) + (9.058528638500256 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7840969804278382 = (1419.2920435308708 - 306.4294378530592) / 1419.2920435308708 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.86732292700692 < 54.38950315112139.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 164.43896113826767 (T) = (0 -19.956697682848386) / Math.Sqrt((3.3968265540688036 / (70)) + (0.23704732984175508 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.6604210138729693 = (58.768941831350325 - 19.956697682848386) / 58.768941831350325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.265782872413844 < 89.55434279948113.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 101.4204574013752 (T) = (0 -33.566841978599555) / Math.Sqrt((28.278601471133072 / (70)) + (0.3223520830056374 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.6602676799272026 = (98.80379344363496 - 33.566841978599555) / 98.80379344363496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFourValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.40720993033277 < 121.72872019775446.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 69.60968337038388 (T) = (0 -55.01093879973165) / Math.Sqrt((78.61031028025636 / (70)) + (0.4785506155002254 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.5743913490127981 = (129.252398117692 - 55.01093879973165) / 129.252398117692 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 174.33117651696023 < 674.6076948435476.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 243.18653473008987 (T) = (0 -186.41630998862948) / Math.Sqrt((100.15603086391059 / (299)) + (146.87788926839093 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.740520297897375 = (718.4234777443255 - 186.41630998862948) / 718.4234777443255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 413.07344968444903 < 1.937959543494133.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 669.7005399910396 (T) = (0 -409.096203616456) / Math.Sqrt((1266.717468848586 / (299)) + (55.01796637412804 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7990989068200198 = (2036.3065085462788 - 409.096203616456) / 2036.3065085462788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.192134834371632 < 12.58788240164642.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.328769436312879 (T) = (0 -10.142830619753067) / Math.Sqrt((8.05101669535518 / (69)) + (0.0010362831651648783 / (33))) is greater than 1.9839715185231144 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (69) + (33) - 2, .975) and 0.23908845738014747 = (13.32984197457545 - 10.142830619753067) / 13.32984197457545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 308.3170700792926 < 1.195713752511473.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 320.7585698497216 (T) = (0 -305.6076612611339) / Math.Sqrt((2644.452146226993 / (299)) + (16.106224921703017 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7622639953930959 = (1285.4917023042233 - 305.6076612611339) / 1285.4917023042233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 507.5145831398664 < 1.4453985137698675.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 186.447164679776 (T) = (0 -532.0116033181567) / Math.Sqrt((5693.972288558635 / (299)) + (709.397564937892 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.6905369195576795 = (1719.1440173016574 - 532.0116033181567) / 1719.1440173016574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyThreeValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.23622700235071 < 251.01114215239676.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 456.6067423971539 (T) = (0 -43.08968370227705) / Math.Sqrt((16.069183923477066 / (70)) + (0.10477924497767603 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.8363902465767334 = (263.36867332598376 - 43.08968370227705) / 263.36867332598376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 191.36133900038598 < 762.180029967062.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 367.9162855432756 (T) = (0 -203.3811837340004) / Math.Sqrt((527.8153034852452 / (299)) + (31.723911719052463 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.7491907664901062 = (810.8999054294287 - 203.3811837340004) / 810.8999054294287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFourValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.615454576352573 < 29.031144242450196.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 18.637187640039528 (T) = (0 -27.900056001495486) / Math.Sqrt((0.07419034729517486 / (70)) + (0.6292329182353404 / (32))) is greater than 1.9839715185231144 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (32) - 2, .975) and 0.08772649409486759 = (30.582994925204833 - 27.900056001495486) / 30.582994925204833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyTwoValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.83327707316224 < 177.6236149015865.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 322.23179715681306 (T) = (0 -33.66998833293706) / Math.Sqrt((16.071656972611347 / (70)) + (0.09400732344299155 / (33))) is greater than 1.9837310029553383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (33) - 2, .975) and 0.8218761879206719 = (189.0257565223341 - 33.66998833293706) / 189.0257565223341 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf - Duration of single invocation 23.78 ns 14.51 ns 0.61 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.506627232553065 < 22.598804904530933.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 182.19564615484109 (T) = (0 -14.567608037433532) / Math.Sqrt((0.032049361058740004 / (299)) + (0.0828811788362375 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.39025262075235984 = (23.89121877884629 - 14.567608037433532) / 23.89121877884629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCredential_Uri - Duration of single invocation 530.93 ns 271.25 ns 0.51 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 271.2456461820833 < 468.4614856432857.
IsChangePoint: Marked as a change because one of 11/23/2022 1:02:40 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 9.044638718747384 (T) = (0 -269.5901083315688) / Math.Sqrt((17513.629721397374 / (299)) + (14.83893582623169 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.20493008579288963 = (339.07723523964506 - 269.5901083315688) / 339.07723523964506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 383.92 ns 50.15 ns 0.13 0.09 False
IndexOfString - Duration of single invocation 317.78 ns 48.35 ns 0.15 0.08 False
IndexOfString - Duration of single invocation 3.58 μs 401.44 ns 0.11 0.08 False
IndexOfString - Duration of single invocation 1.11 μs 133.58 ns 0.12 0.10 False
IndexOfString - Duration of single invocation 3.59 μs 401.19 ns 0.11 0.08 False
IndexOfString - Duration of single invocation 1.33 μs 155.84 ns 0.12 0.06 False
IndexOfString - Duration of single invocation 152.85 ns 27.53 ns 0.18 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.14854979160225 < 364.8514460874174.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 411.64982727958665 (T) = (0 -49.82463619530422) / Math.Sqrt((201.72919948064327 / (299)) + (0.21984034614757747 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8721180945723539 = (389.61443394737614 - 49.82463619530422) / 389.61443394737614 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.35161689482309 < 302.1940954026377.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 441.2069995223351 (T) = (0 -48.22286353617805) / Math.Sqrt((104.51343930213636 / (299)) + (1.0380225874231543 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8495659746288033 = (320.55822090240514 - 48.22286353617805) / 320.55822090240514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 401.4438114893914 < 3.420342033315042.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 390.70769584435294 (T) = (0 -400.6014682191244) / Math.Sqrt((20578.385332966503 / (299)) + (0.6957032171768623 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8900177319921118 = (3642.4186868958936 - 400.6014682191244) / 3642.4186868958936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 133.58467936643834 < 1.0568511096430793.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 292.4532135534877 (T) = (0 -129.87847147912055) / Math.Sqrt((2899.3796996465853 / (299)) + (16.084097115233902 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.877839724183444 = (1063.180895843382 - 129.87847147912055) / 1063.180895843382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 401.18553496682995 < 3.4278223284408944.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 396.85293357546806 (T) = (0 -400.6274034290944) / Math.Sqrt((19897.352768036217 / (299)) + (0.7315270246741807 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8898931038838326 = (3638.531441358729 - 400.6274034290944) / 3638.531441358729 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 155.84359457006528 < 1.2663911630801574.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 464.01234692958366 (T) = (0 -155.20419381232225) / Math.Sqrt((2000.1296510636796 / (299)) + (0.8664326817826988 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8856836289810722 = (1357.6725050747498 - 155.20419381232225) / 1357.6725050747498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.533916137379435 < 145.2752764356027.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 315.2857276319589 (T) = (0 -27.103271336432645) / Math.Sqrt((40.636742936104596 / (299)) + (0.4658500606994679 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.8183710382940125 = (149.2232906132346 - 27.103271336432645) / 149.2232906132346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileNameWithoutExtension - Duration of single invocation 141.50 ns 72.75 ns 0.51 0.02 False
GetFileName - Duration of single invocation 127.21 ns 65.56 ns 0.52 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.74725568990382 < 134.3185567551144.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 366.68007408695 (T) = (0 -73.10333554039211) / Math.Sqrt((8.314183008113773 / (299)) + (0.35506369636771506 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4973164560727865 = (145.42615612453224 - 73.10333554039211) / 145.42615612453224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFileName

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.56115131381122 < 118.81806931959063.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsImprovementStdDev: Marked as improvement because 291.8975221298311 (T) = (0 -65.4732249723077) / Math.Sqrt((6.668999153589068 / (299)) + (0.8258178842487238 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.4923623051225894 = (128.97628689319225 - 65.4732249723077) / 128.97628689319225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Jan 10, 2023

dotnet/runtime#79215 again

cc @SamMonoRT @BrzVlad

@naricc naricc closed this as completed Jan 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-aot os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @naricc