Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regressions in Perf_String, Perf_Stringbuilder, other on 8/9/2022 #77064

Closed
performanceautofiler bot opened this issue Aug 16, 2022 · 11 comments
Closed

Regressions in Perf_String, Perf_Stringbuilder, other on 8/9/2022 #77064

performanceautofiler bot opened this issue Aug 16, 2022 · 11 comments
Assignees
@kunalspathak
Labels
arch-x64 area-System.Runtime runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Remove_IntInt - Duration of single invocation 11.88 ns 13.46 ns 1.13 0.04 False
Trim_CharArr - Duration of single invocation 15.72 ns 17.44 ns 1.11 0.03 False
Insert - Duration of single invocation 14.84 ns 16.75 ns 1.13 0.03 False
Substring_Int - Duration of single invocation 7.71 ns 9.22 ns 1.20 0.10 False
CtorCharCount - Duration of single invocation 9.63 ns 11.28 ns 1.17 0.06 False
Trim_CharArr - Duration of single invocation 15.14 ns 16.58 ns 1.09 0.03 False
TrimStart_CharArr - Duration of single invocation 13.07 ns 15.04 ns 1.15 0.04 False
Trim_CharArr - Duration of single invocation 18.13 ns 19.37 ns 1.07 0.04 False
GetChars - Duration of single invocation 16.15 ns 18.18 ns 1.13 0.04 False
Replace_Char - Duration of single invocation 25.73 ns 27.68 ns 1.08 0.04 False
Substring_IntInt - Duration of single invocation 7.61 ns 9.43 ns 1.24 0.09 False
Insert - Duration of single invocation 15.24 ns 17.36 ns 1.14 0.03 False
PadLeft - Duration of single invocation 12.05 ns 13.09 ns 1.09 0.15 False
Replace_Char - Duration of single invocation 21.17 ns 23.10 ns 1.09 0.02 False
Substring_IntInt - Duration of single invocation 7.83 ns 9.84 ns 1.26 0.07 False
Remove_IntInt - Duration of single invocation 12.06 ns 13.85 ns 1.15 0.04 False
CtorCharCount - Duration of single invocation 7.96 ns 9.85 ns 1.24 0.08 False
TrimEnd_CharArr - Duration of single invocation 14.36 ns 15.62 ns 1.09 0.05 False
Substring_Int - Duration of single invocation 7.76 ns 9.15 ns 1.18 0.12 False
Replace_String - Duration of single invocation 28.54 ns 30.04 ns 1.05 0.03 False
Substring_IntInt - Duration of single invocation 8.39 ns 10.07 ns 1.20 0.09 False
Remove_IntInt - Duration of single invocation 12.31 ns 13.72 ns 1.11 0.06 False
CtorCharCount - Duration of single invocation 16.91 ns 18.82 ns 1.11 0.06 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.458880221615381 > 12.323685492321225.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -63.05248742051773 (T) = (0 -13.35949958004901) / Math.Sqrt((0.008845259925720085 / (20)) + (0.011335104513203001 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.14565549198003522 = (11.661009503790533 - 13.35949958004901) / 11.661009503790533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.442991300883428 > 16.389468749734654.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 3:55:24 AM, 7/26/2022 5:16:37 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -17.53574088487783 (T) = (0 -17.57255894206272) / Math.Sqrt((0.13017317560899197 / (20)) + (0.028604759004090628 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09267493259124028 = (16.082147048427306 - 17.57255894206272) / 16.082147048427306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.745340494409 > 15.7702016065585.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/4/2022 6:29:12 PM, 7/6/2022 1:39:15 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -8.09343774740642 (T) = (0 -16.688767663312245) / Math.Sqrt((0.2188977269711232 / (20)) + (0.006546919949689709 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.05386708472754068 = (15.835742386457435 - 16.688767663312245) / 15.835742386457435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.223322921851516 > 7.792330455508882.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -36.70253957161099 (T) = (0 -9.672184263775653) / Math.Sqrt((0.008598152621522555 / (20)) + (0.12233827470101226 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.2888607042548379 = (7.50444499692283 - 9.672184263775653) / 7.50444499692283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.28234880445019 > 10.29204619995384.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -40.20782312602304 (T) = (0 -11.373944422243357) / Math.Sqrt((0.02384996979671384 / (20)) + (0.002811354960092524 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.14366937585733444 = (9.945133324670035 - 11.373944422243357) / 9.945133324670035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.575885270328836 > 15.516340913481352.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 1:39:15 AM, 7/28/2022 3:42:13 AM, 8/1/2022 6:03:22 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -9.45104317668793 (T) = (0 -16.94481879422906) / Math.Sqrt((0.3427480328011729 / (20)) + (0.08191045734979938 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.08384689591350462 = (15.633959794614134 - 16.94481879422906) / 15.633959794614134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.035952923753275 > 13.698417077458656.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 1:39:15 AM, 7/24/2022 8:36:55 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -11.682605554450639 (T) = (0 -15.013609700812694) / Math.Sqrt((0.22908401492976385 / (20)) + (0.0145648323958909 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09240981170912917 = (13.743569070771306 - 15.013609700812694) / 13.743569070771306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: " Te st  ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.37070633908783 > 18.96510022255503.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 6/30/2022 4:27:12 PM, 7/7/2022 8:07:51 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -29.72842177195503 (T) = (0 -19.62300556564896) / Math.Sqrt((0.039357702102435256 / (18)) + (0.039999662268168736 / (40))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (40) - 2, .025) and -0.09351708092949 = (17.944855099080296 - 19.62300556564896) / 17.944855099080296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.GetChars(size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.184019786236682 > 17.103696802551763.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -53.1012790865293 (T) = (0 -18.27982243488964) / Math.Sqrt((0.025615712287388893 / (20)) + (0.016321472072452943 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.13556125049336984 = (16.097610258317253 - 18.27982243488964) / 16.097610258317253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.67953214612285 > 26.211930390726824.
IsChangePoint: Marked as a change because one of 7/7/2022 8:07:51 AM, 7/10/2022 1:04:01 AM, 7/13/2022 1:49:45 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.37369442086753 (T) = (0 -27.52857211363341) / Math.Sqrt((0.3986877818782333 / (20)) + (0.04998724862440364 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.13416350297379895 = (24.27213716668977 - 27.52857211363341) / 24.27213716668977 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.426734291823244 > 7.9117523499889435.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 12:32:56 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -40.449662534535655 (T) = (0 -9.37004646612724) / Math.Sqrt((0.054833057708211695 / (20)) + (0.01400185947370848 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.3158474071842175 = (7.120921783915817 - 9.37004646612724) / 7.120921783915817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.360342888810553 > 16.63104364207198.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -15.354982829432739 (T) = (0 -17.289014424679973) / Math.Sqrt((0.11169264642952915 / (18)) + (0.008356415597080974 / (40))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (40) - 2, .025) and -0.07657553246351055 = (16.059267467390605 - 17.289014424679973) / 16.059267467390605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.PadLeft(n: 18)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.094671984522298 > 11.998458424158024.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -16.814737755413415 (T) = (0 -13.781697574207982) / Math.Sqrt((0.11980002360747036 / (20)) + (0.37176288283189796 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1776297510001823 = (11.702912195028137 - 13.781697574207982) / 11.702912195028137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.098672440738362 > 21.90359221421459.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -10.17941463137221 (T) = (0 -23.075328685809435) / Math.Sqrt((0.42473609067567436 / (7)) + (0.017770962796527304 / (40))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (40) - 2, .025) and -0.12241134795814908 = (20.55870936068782 - 23.075328685809435) / 20.55870936068782 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.843047748606525 > 8.205400107385309.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -24.88788601176509 (T) = (0 -9.922853850421653) / Math.Sqrt((0.08297911298044891 / (20)) + (0.2832283316313345 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.36200378167964936 = (7.285481864216704 - 9.922853850421653) / 7.285481864216704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.847245706923978 > 12.784272639740891.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -39.55151525449711 (T) = (0 -13.956003830970548) / Math.Sqrt((0.007689134819444583 / (7)) + (0.03611117269714392 / (40))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (40) - 2, .025) and -0.14518658544241028 = (12.186663735306542 - 13.956003830970548) / 12.186663735306542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.850974994977735 > 8.372996448790467.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -31.99966318274964 (T) = (0 -9.893426406232269) / Math.Sqrt((0.019631505017476288 / (20)) + (0.08239181122689478 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.21709986940849477 = (8.128689070552962 - 9.893426406232269) / 8.128689070552962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.616962333457307 > 15.08212603434616.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 6/30/2022 4:27:12 PM, 7/6/2022 3:55:24 AM, 7/26/2022 6:20:16 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.143272393779462 (T) = (0 -15.666648509497207) / Math.Sqrt((0.0531031761327196 / (17)) + (0.021396580008356952 / (40))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (40) - 2, .025) and -0.0934840325694977 = (14.327276890073376 - 15.666648509497207) / 14.327276890073376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.14664838078872 > 7.866964388609689.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -34.8772616286586 (T) = (0 -9.66122102625102) / Math.Sqrt((0.016226408411900234 / (20)) + (0.11677215103087843 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.2828673858697124 = (7.530958486173731 - 9.66122102625102) / 7.530958486173731 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.0424995318874 > 29.45401414871855.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.345187203220288 (T) = (0 -30.09628951628173) / Math.Sqrt((0.08341998465796059 / (20)) + (0.2692124876551383 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.08403363967062433 = (27.76324314568897 - 30.09628951628173) / 27.76324314568897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.065460869195261 > 8.863186282537988.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -34.09494349666231 (T) = (0 -10.47691481750297) / Math.Sqrt((0.029679385710329825 / (20)) + (0.10637482543009164 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.26498185835553084 = (8.282264878583238 - 10.47691481750297) / 8.282264878583238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.72135757462028 > 12.897907381147377.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -13.106291726793815 (T) = (0 -13.895130048375464) / Math.Sqrt((0.054250772856808134 / (20)) + (0.2615186672720924 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09977046482956478 = (12.634572842914853 - 13.895130048375464) / 12.634572842914853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.81634604031629 > 17.708728079417345.
IsChangePoint: Marked as a change because one of 6/21/2022 5:13:43 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.779207881011875 (T) = (0 -18.79626944340756) / Math.Sqrt((0.2756523566834548 / (20)) + (0.011766423251870505 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1766074178431023 = (15.974971055225827 - 18.79626944340756) / 15.974971055225827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringHex - Duration of single invocation 19.28 ns 20.34 ns 1.06 0.06 False
ToString - Duration of single invocation 11.09 ns 12.27 ns 1.11 0.07 False
ToStringHex - Duration of single invocation 25.42 ns 27.64 ns 1.09 0.03 False
ToString - Duration of single invocation 16.97 ns 18.27 ns 1.08 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToStringHex(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.3438823277552 > 19.35999656916104.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -19.662082462612894 (T) = (0 -20.399557788406188) / Math.Sqrt((0.11301839374618175 / (20)) + (0.1023318492346544 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09568535108814727 = (18.618080243700415 - 20.399557788406188) / 18.618080243700415 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.265364745407124 > 11.578409535243326.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/2/2022 3:52:19 AM, 7/4/2022 6:29:12 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -32.95671667775704 (T) = (0 -12.425452634575908) / Math.Sqrt((0.009606591167517048 / (20)) + (0.04130679073919822 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.11615651328518213 = (11.132356875295283 - 12.425452634575908) / 11.132356875295283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.638065026688533 > 25.665333977691123.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -26.422305010768586 (T) = (0 -27.39829691503944) / Math.Sqrt((0.15747210833792727 / (20)) + (0.07024077056765961 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.10452691086761844 = (24.80545892133834 - 27.39829691503944) / 24.80545892133834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.26866092072727 > 17.87855185425306.
IsChangePoint: Marked as a change because one of 7/2/2022 3:52:19 AM, 7/4/2022 6:29:12 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -29.981702514702484 (T) = (0 -18.34288502955733) / Math.Sqrt((0.017645643245940756 / (20)) + (0.025699448514064045 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.06817631586004307 = (17.17215103649676 - 18.34288502955733) / 17.17215103649676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added CoreClr untriaged New issue has not been triaged by the area owner labels Aug 16, 2022
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadToEnd - Duration of single invocation 68.30 ms 80.77 ms 1.18 0.08 False
ReadToEnd - Duration of single invocation 67.71 ms 81.51 ms 1.20 0.09 False
ReadToEnd - Duration of single invocation 73.77 ms 82.73 ms 1.12 0.12 False
ReadToEnd - Duration of single invocation 70.45 ms 80.85 ms 1.15 0.06 False
ReadToEndAsync - Duration of single invocation 74.37 ms 84.74 ms 1.14 0.05 False
ReadToEndAsync - Duration of single invocation 73.65 ms 86.64 ms 1.18 0.03 False
ReadToEndAsync - Duration of single invocation 75.59 ms 87.26 ms 1.15 0.06 False
ReadToEndAsync - Duration of single invocation 75.15 ms 85.88 ms 1.14 0.02 False
ReadToEndAsync - Duration of single invocation 72.90 ms 84.91 ms 1.16 0.05 False
ReadToEndAsync - Duration of single invocation 72.37 ms 84.53 ms 1.17 0.04 False
ReadToEnd - Duration of single invocation 61.10 ms 82.22 ms 1.35 0.08 False
ReadToEndAsync - Duration of single invocation 73.84 ms 84.70 ms 1.15 0.04 False
ReadToEndAsync - Duration of single invocation 72.78 ms 84.70 ms 1.16 0.04 False
ReadToEnd - Duration of single invocation 67.64 ms 81.53 ms 1.21 0.17 False
ReadToEnd - Duration of single invocation 67.46 ms 80.96 ms 1.20 0.09 False
ReadToEnd - Duration of single invocation 72.37 ms 80.61 ms 1.11 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.77020625 > 73.78482759375001.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -21.323621947558596 (T) = (0 -82694636.91029033) / Math.Sqrt((2831437642585.308 / (20)) + (4722088787345.626 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15126218633845834 = (71829543.16713658 - 82694636.91029033) / 71829543.16713658 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.5064525 > 73.64820825721155.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -19.387496380786832 (T) = (0 -82811042.30119213) / Math.Sqrt((4254525669133.8423 / (20)) + (3880114788702.529 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.149813886527743 = (72021257.7630876 - 82811042.30119213) / 72021257.7630876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.7282675 > 75.03572008717106.
IsChangePoint: Marked as a change because one of 6/24/2022 7:16:38 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -17.90059951352489 (T) = (0 -82387185.92167217) / Math.Sqrt((6090045678301.497 / (20)) + (3725953038945.859 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15876483397497262 = (71099142.3851335 - 82387185.92167217) / 71099142.3851335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.84885921052631 > 76.20886.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.12404297550215 (T) = (0 -82288003.03748877) / Math.Sqrt((1711737685769.0293 / (20)) + (4137978665830.121 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.13918563217037794 = (72234059.76488096 - 82288003.03748877) / 72234059.76488096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.73949125 > 77.30388116099073.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -19.812401677930332 (T) = (0 -84649074.02514288) / Math.Sqrt((2079779763087.812 / (20)) + (7800167803171.268 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.14676402599631572 = (73815599.46615803 - 84649074.02514288) / 73815599.46615803 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.6406475 > 77.055266875.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.646125697205992 (T) = (0 -85052649.1185059) / Math.Sqrt((1750618752593.8066 / (20)) + (5135952254370.025 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.14835592637642803 = (74064710.39591767 - 85052649.1185059) / 74064710.39591767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.25644125 > 78.355974375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.378269031509763 (T) = (0 -85713156.48027426) / Math.Sqrt((2282427798623.4233 / (20)) + (5690258767671.672 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15233431943248218 = (74382195.37059999 - 85713156.48027426) / 74382195.37059999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.87856625 > 77.9997054375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -28.02254299193879 (T) = (0 -85573693.46600279) / Math.Sqrt((1319848416144.8394 / (20)) + (4276643809977.7964 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1576326707451178 = (73921284.0381593 - 85573693.46600279) / 73921284.0381593 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.9145725 > 76.56765857536766.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -21.730810847595567 (T) = (0 -85278338.2318937) / Math.Sqrt((1565954826102.1194 / (20)) + (8462682444461.103 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15900901965074393 = (73578666.5901802 - 85278338.2318937) / 73578666.5901802 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.53191666666667 > 76.68191851214576.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.81461978342102 (T) = (0 -85233946.77083024) / Math.Sqrt((1179362380733.3076 / (20)) + (8144723301453.846 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15896728795021572 = (73543013.38528506 - 85233946.77083024) / 73543013.38528506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.2211675 > 75.98277617487982.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -13.661878777406947 (T) = (0 -82707284.94271502) / Math.Sqrt((10024298920803.393 / (20)) + (6722877408480.199 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15625188825665354 = (71530507.99979015 - 82707284.94271502) / 71530507.99979015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.7039275 > 77.072656828125.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.505329011429964 (T) = (0 -85406731.96987528) / Math.Sqrt((1873574890547.706 / (20)) + (5640376126122.035 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15383841751368157 = (74019663.99585804 - 85406731.96987528) / 74019663.99585804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.70488625 > 76.50531648574564.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -24.77345389253488 (T) = (0 -85592073.44428568) / Math.Sqrt((1679889272263.1902 / (20)) + (5775695931458.805 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1605251686056793 = (73752879.95444411 - 85592073.44428568) / 73752879.95444411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.533505 > 72.4829634375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -21.016104966639194 (T) = (0 -82527801.29536732) / Math.Sqrt((4091175155332.5635 / (20)) + (3839355510235.72 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.16225862192446772 = (71006400.58812194 - 82527801.29536732) / 71006400.58812194 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.963285 > 75.88075643750001.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -20.514412862825157 (T) = (0 -82081323.98510948) / Math.Sqrt((3111385372605.746 / (20)) + (2943591076430.9126 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.135901380483591 = (72260959.79403135 - 82081323.98510948) / 72260959.79403135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.6120346153846 > 76.3044459423077.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -25.93598402592468 (T) = (0 -82434740.28015542) / Math.Sqrt((27820226563.018 / (20)) + (5622293696496.905 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1344788687277442 = (72663090.12225275 - 82434740.28015542) / 72663090.12225275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 11.24 ns 12.79 ns 1.14 0.08 False
ToString - Duration of single invocation 17.51 ns 18.73 ns 1.07 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.789902515428512 > 12.031041686016268.
IsChangePoint: Marked as a change because one of 6/17/2022 8:24:28 PM, 7/10/2022 1:04:01 AM, 7/13/2022 7:22:00 AM, 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -12.120117913634413 (T) = (0 -12.56298950645112) / Math.Sqrt((0.04345859429039118 / (20)) + (0.10397402745749361 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.07140532939463491 = (11.7257112334409 - 12.56298950645112) / 11.7257112334409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ToString(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.729530721935017 > 18.30694562515458.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -37.09790889089997 (T) = (0 -18.78371724803076) / Math.Sqrt((0.01573214285767957 / (20)) + (0.014311948227680995 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.0715960436179871 = (17.528729561759153 - 18.78371724803076) / 17.528729561759153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IFooObjIsIFoo - Duration of single invocation 417.91 μs 501.18 μs 1.20 0.34 False
FooObjIsDescendant - Duration of single invocation 411.54 μs 501.18 μs 1.22 0.23 False
FooObjIsFoo - Duration of single invocation 489.59 μs 612.56 μs 1.25 0.51 False
ObjFooIsObj - Duration of single invocation 423.33 μs 557.19 μs 1.32 0.57 False
IFooObjIsDescendantOfIFoo - Duration of single invocation 417.76 μs 501.19 μs 1.20 0.35 False
IFooFooIsIFoo - Duration of single invocation 334.04 μs 556.99 μs 1.67 0.54 False
FooObjCastIfIsa - Duration of single invocation 306.31 μs 417.72 μs 1.36 0.48 False
IFooObjIsIFooInterAlia - Duration of single invocation 361.93 μs 417.84 μs 1.15 0.48 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFoo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 501.18244977678575 > 450.3861866618169.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -5.577121934358572 (T) = (0 -501428.5926562858) / Math.Sqrt((903020040.8608952 / (20)) + (70502.94204552045 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.08077530989473886 = (463952.67181401653 - 501428.5926562858) / 463952.67181401653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.FooObjIsDescendant

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 501.18182198660713 > 435.299202155027.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 7/6/2022 12:32:56 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -35.856133627453204 (T) = (0 -501297.26702813787) / Math.Sqrt((110381056.2729788 / (20)) + (99644.91291009837 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.20202878655471584 = (417042.6470941418 - 501297.26702813787) / 417042.6470941418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.FooObjIsFoo

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 612.5565590659342 > 516.966491399397.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -17.487467653845023 (T) = (0 -612577.3917234609) / Math.Sqrt((1141961002.1647615 / (20)) + (8778.549347854465 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.27504434700087355 = (480436.14574218506 - 612577.3917234609) / 480436.14574218506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjFooIsObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 557.188263248848 > 426.2831046947122.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -16.635324373038564 (T) = (0 -556988.8594940526) / Math.Sqrt((2817797114.7486362 / (20)) + (32701.400482233017 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.5492055771857568 = (359531.9224875648 - 556988.8594940526) / 359531.9224875648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsDescendantOfIFoo

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 501.188671875 > 460.6289412305777.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -11.917615113624164 (T) = (0 -501395.8502369633) / Math.Sqrt((400153109.1374179 / (20)) + (201242.774619666 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.11898296639569898 = (448081.7539626942 - 501395.8502369633) / 448081.7539626942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooFooIsIFoo

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 556.9870103686636 > 350.8029813062602.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -18.658258933907295 (T) = (0 -556983.3157225405) / Math.Sqrt((2479225098.077693 / (20)) + (31055.157967016443 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.5948186508552369 = (349245.5492816395 - 556983.3157225405) / 349245.5492816395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.FooObjCastIfIsa

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 417.7226785714285 > 373.9503897299509.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -10.31220448222303 (T) = (0 -417781.26800151996) / Math.Sqrt((660720300.7307487 / (20)) + (23801.148256836743 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.16532891416248 = (358509.312627653 - 417781.26800151996) / 358509.312627653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 417.8382346491228 > 359.3401925499613.
IsChangePoint: Marked as a change because one of 6/16/2022 10:20:28 PM, 6/27/2022 8:24:30 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -9.139312151149301 (T) = (0 -417738.02664918) / Math.Sqrt((879521643.2132629 / (20)) + (19181.331150208487 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.16970584273367137 = (357130.83699137694 - 417738.02664918) / 357130.83699137694 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Collections.CtorDefaultSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentStack - Duration of single invocation 2.88 ns 4.01 ns 1.39 0.36 False
Dictionary - Duration of single invocation 14.46 ns 15.70 ns 1.09 0.11 False
HashSet - Duration of single invocation 14.64 ns 15.86 ns 1.08 0.08 False
Stack - Duration of single invocation 12.12 ns 13.31 ns 1.10 0.10 False
List - Duration of single invocation 9.95 ns 11.03 ns 1.11 0.08 False
LinkedList - Duration of single invocation 3.12 ns 4.71 ns 1.51 0.25 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorDefaultSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<String>.ConcurrentStack

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.009501707156229 > 2.9460506503725545.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -50.90092498835992 (T) = (0 -3.992118191033332) / Math.Sqrt((0.006733639209554932 / (20)) + (0.00854269705390884 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.4267169420001943 = (2.7981150805124373 - 3.992118191033332) / 2.7981150805124373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.Dictionary

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.698367658774988 > 15.157722046987576.
IsChangePoint: Marked as a change because one of 7/9/2022 2:25:16 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.735528960694392 (T) = (0 -15.985219333345546) / Math.Sqrt((0.11212360684061455 / (20)) + (0.0622041770484231 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.14371178480222463 = (13.976615040396542 - 15.985219333345546) / 13.976615040396542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.HashSet

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.856220760756361 > 15.239171267960726.
IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -14.666584949138413 (T) = (0 -16.22864870057163) / Math.Sqrt((0.037670990134063186 / (20)) + (0.598954449964847 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.13293768110048507 = (14.324396629484372 - 16.22864870057163) / 14.324396629484372 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.Stack

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.314017628695401 > 12.764077429043894.
IsChangePoint: Marked as a change because one of 7/28/2022 12:19:57 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -10.168563812230728 (T) = (0 -13.290623272364865) / Math.Sqrt((0.15015404717152842 / (20)) + (0.05899193661697863 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.07818165123287905 = (12.32688689996468 - 13.290623272364865) / 12.32688689996468 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.List

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.026642321593862 > 10.321218368201512.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -3.711177062578812 (T) = (0 -11.18698012128837) / Math.Sqrt((1.0725751671924348 / (20)) + (0.04270565564368074 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.08411081819499011 = (10.319037439285344 - 11.18698012128837) / 10.319037439285344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.LinkedList

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.710285223582388 > 3.381492298293707.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -65.49944649139769 (T) = (0 -4.665168099539699) / Math.Sqrt((0.007393988892487168 / (20)) + (0.00345623113865782 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.4282636848728261 = (3.2663213025367157 - 4.665168099539699) / 3.2663213025367157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Object

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ctor - Duration of single invocation 1.95 ns 3.44 ns 1.77 0.46 False
GetType_ - Duration of single invocation 0.84 ns 2.23 ns 2.66 0.71 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Object*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Object.ctor

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4386677295397607 > 2.364090878203656.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -48.20516931987895 (T) = (0 -3.410726533514712) / Math.Sqrt((0.008438381182055945 / (20)) + (0.007485576698018643 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.5356260290723224 = (2.221065851283561 - 3.410726533514712) / 2.221065851283561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Object.GetType_

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2273431750292216 > 1.1708504179714319.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -17.080042824530185 (T) = (0 -2.1257774425303992) / Math.Sqrt((0.051548182440423806 / (20)) + (0.017756180126864876 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.7909614048684299 = (1.186947656578097 - 2.1257774425303992) / 1.186947656578097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in PerfLabTests.EnumPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ObjectGetTypeNoBoxing - Duration of single invocation 0.84 ns 2.23 ns 2.66 0.68 False
ObjectGetType - Duration of single invocation 0.84 ns 1.95 ns 2.34 0.64 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.EnumPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.EnumPerf.ObjectGetTypeNoBoxing

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2284719264508603 > 1.4818239003006233.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -12.240928466046523 (T) = (0 -2.123448492544857) / Math.Sqrt((0.09468128474275168 / (20)) + (0.018528819813095294 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.7111156807570164 = (1.2409730776386902 - 2.123448492544857) / 1.2409730776386902 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.EnumPerf.ObjectGetType

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9530820351425204 > 1.4618452990510056.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -9.204576471386561 (T) = (0 -1.9838467656979568) / Math.Sqrt((0.08211756651854225 / (20)) + (0.00870060668953965 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.43900631647507166 = (1.3786226946921978 - 1.9838467656979568) / 1.3786226946921978 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString_SingleSegment - Duration of single invocation 10.86 μs 11.84 μs 1.09 0.01 False
Append_Char_Capacity - Duration of single invocation 177.26 μs 220.70 μs 1.25 0.64 False
ToString_MultipleSegments - Duration of single invocation 11.18 μs 12.09 μs 1.08 0.00 False
ToString_SingleSegment - Duration of single invocation 17.63 ns 19.59 ns 1.11 0.03 False
ctor_capacity - Duration of single invocation 3.90 μs 4.12 μs 1.05 0.05 False
ctor_string - Duration of single invocation 25.68 ns 27.72 ns 1.08 0.03 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.835080669526858 > 11.397833166908079.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -38.23826278553131 (T) = (0 -11865.484127582327) / Math.Sqrt((2968.803964340659 / (20)) + (20893.644026907605 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09106552534768322 = (10875.134308547807 - 11865.484127582327) / 10875.134308547807 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 220.69660211267603 > 186.4418998428781.
IsChangePoint: Marked as a change because one of 7/6/2022 9:44:27 PM, 7/13/2022 11:09:36 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -5.103224567970274 (T) = (0 -216064.4355543404) / Math.Sqrt((601855435.2436146 / (20)) + (125905598.73289368 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15764070936209473 = (186642.05034168187 - 216064.4355543404) / 186642.05034168187 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.094558563817051 > 11.665420159214218.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -53.18685992684513 (T) = (0 -12141.525182218222) / Math.Sqrt((1412.746147552604 / (20)) + (11363.9984197381 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.0899251539642734 = (11139.77885367366 - 12141.525182218222) / 11139.77885367366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.585462644081726 > 18.418945293301707.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -24.547192889592502 (T) = (0 -19.559923258388192) / Math.Sqrt((0.1866462739016429 / (17)) + (0.02291920270729478 / (40))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (40) - 2, .025) and -0.15591687765308992 = (16.921565587052925 - 19.559923258388192) / 16.921565587052925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.117816075321718 > 4.1121432951770265.
IsChangePoint: Marked as a change because one of 6/18/2022 12:36:47 AM, 6/22/2022 10:05:16 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -12.2094705985269 (T) = (0 -4191.6763120508795) / Math.Sqrt((4914.246392895704 / (20)) + (8294.512644313902 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.06609848993597858 = (3931.7908726262226 - 4191.6763120508795) / 3931.7908726262226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.722962531613124 > 26.069209632497852.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -25.290345273352546 (T) = (0 -27.354011293590474) / Math.Sqrt((0.1616307711596807 / (18)) + (0.0858177848417178 / (40))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (40) - 2, .025) and -0.10805468778216097 = (24.68651736706353 - 27.354011293590474) / 24.68651736706353 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Reflection.Activator<EmptyClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateInstanceGeneric - Duration of single invocation 12.26 ns 13.74 ns 1.12 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Activator&lt;EmptyClass&gt;*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Activator<EmptyClass>.CreateInstanceGeneric

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.74254281126468 > 12.774697275781831.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -40.64222870779937 (T) = (0 -13.657874858399964) / Math.Sqrt((0.019103908877144492 / (20)) + (0.017953197336368555 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.12549467873770057 = (12.134997274014626 - 13.657874858399964) / 12.134997274014626 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 11.09 ns 12.52 ns 1.13 0.06 False
ToString - Duration of single invocation 11.12 ns 12.52 ns 1.13 0.07 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.524955547386242 > 11.597444267079307.
IsChangePoint: Marked as a change because one of 6/17/2022 8:24:28 PM, 6/30/2022 4:27:12 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -43.26359048434788 (T) = (0 -12.446623016051111) / Math.Sqrt((0.011830006162302991 / (20)) + (0.02760397381896391 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1421214022448774 = (10.897810855822211 - 12.446623016051111) / 10.897810855822211 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.ToString(value: 65535)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.520217927651961 > 11.545219401224239.
IsChangePoint: Marked as a change because one of 6/17/2022 8:24:28 PM, 6/30/2022 4:27:12 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -6.684398162674869 (T) = (0 -12.7073112829914) / Math.Sqrt((0.007819639032562747 / (20)) + (2.9224677111899635 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.16626895409158698 = (10.895695404058142 - 12.7073112829914) / 10.895695404058142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BenchBitOps - Duration of single invocation 580.99 ms 708.71 ms 1.22 0.02 False
BenchLUDecomp - Duration of single invocation 871.51 ms 920.12 ms 1.06 0.00 False
BenchStringSort - Duration of single invocation 1.20 secs 1.42 secs 1.18 0.03 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchBitOps

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 708.7140315789475 > 616.5627475000001.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -32.41816901205986 (T) = (0 -710829777.541656) / Math.Sqrt((167391741840392.53 / (20)) + (167571706177437.38 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.19277824906214305 = (595944617.6190476 - 710829777.541656) / 595944617.6190476 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchLUDecomp

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 920.1223769230769 > 915.1575019999999.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -38.456390689098285 (T) = (0 -928138114.0906595) / Math.Sqrt((13629467079243.912 / (20)) + (53721865954430.516 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.06264787248295339 = (873420196.9670328 - 928138114.0906595) / 873420196.9670328 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchStringSort

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4165241133333333 > 1.3021189125000001.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -14.767036564167135 (T) = (0 -1407204090.6854396) / Math.Sqrt((1417298289607987.2 / (20)) + (400322393847751.25 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.10420476701204981 = (1274405013.2053843 - 1407204090.6854396) / 1274405013.2053843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Collections.CtorDefaultSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.49 ns 4.98 ns 1.43 0.27 False
HashSet - Duration of single invocation 3.22 ns 4.51 ns 1.40 0.29 False
LinkedList - Duration of single invocation 2.63 ns 3.88 ns 1.47 0.39 False
List - Duration of single invocation 3.65 ns 4.89 ns 1.34 0.24 False
ConcurrentStack - Duration of single invocation 2.12 ns 3.45 ns 1.62 0.41 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorDefaultSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<Int32>.Dictionary

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.981213798201763 > 3.865580582085882.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -45.514402674850544 (T) = (0 -4.952136503951609) / Math.Sqrt((0.010831208858822265 / (20)) + (0.010228894668578853 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.35046622132284977 = (3.6669828728487124 - 4.952136503951609) / 3.6669828728487124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;Int32&gt;.HashSet

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.511037889614 > 3.3131017474004736.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -45.43873240970496 (T) = (0 -4.490639852444023) / Math.Sqrt((0.012612119271187518 / (20)) + (0.005551129772109152 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.39017456329885536 = (3.2302704789733943 - 4.490639852444023) / 3.2302704789733943 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;Int32&gt;.LinkedList

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.875721929655632 > 2.7057968775674457.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -42.740909733747735 (T) = (0 -3.853226843495686) / Math.Sqrt((0.012605633670757205 / (20)) + (0.008075691624301568 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.47055042553395593 = (2.6202616221722432 - 3.853226843495686) / 2.6202616221722432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;Int32&gt;.List

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.890176650188511 > 4.184952598336225.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -11.110206473436996 (T) = (0 -4.92947981939703) / Math.Sqrt((0.06157767604426864 / (20)) + (0.019402968936073153 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.15546975299596266 = (4.266212773303339 - 4.92947981939703) / 4.266212773303339 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;Int32&gt;.ConcurrentStack

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.450878959262179 > 2.2675183356249717.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -43.31157788290957 (T) = (0 -3.413428555189371) / Math.Sqrt((0.011696698977592818 / (20)) + (0.007306272495017639 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.5420676235411181 = (2.2135401217690855 - 3.413428555189371) / 2.2135401217690855 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayBitArrayCtor - Duration of single invocation 16.68 ns 17.77 ns 1.07 0.02 False
BitArrayBitArrayCtor - Duration of single invocation 15.21 ns 16.38 ns 1.08 0.02 False
BitArrayNot - Duration of single invocation 7.62 ns 9.63 ns 1.26 0.11 False
BitArrayNot - Duration of single invocation 1.11 ns 2.41 ns 2.16 0.63 False
BitArrayLengthCtor - Duration of single invocation 10.23 ns 11.53 ns 1.13 0.16 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.770273659171604 > 17.449752675472237.
IsChangePoint: Marked as a change because one of 7/6/2022 12:32:56 PM, 7/27/2022 12:09:41 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -15.65610453763704 (T) = (0 -18.176823073110583) / Math.Sqrt((0.060780325063401676 / (20)) + (0.49710837617085774 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.11996958370634275 = (16.229747073092447 - 18.176823073110583) / 16.229747073092447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayBitArrayCtor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.382368584644418 > 15.839185968071991.
IsChangePoint: Marked as a change because one of 7/27/2022 12:09:41 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.513138920659294 (T) = (0 -16.393196188326183) / Math.Sqrt((0.08717076558954058 / (20)) + (0.02267151208371613 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1119286850408281 = (14.743028405391172 - 16.393196188326183) / 14.743028405391172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.625063943291435 > 8.199670275984628.
IsChangePoint: Marked as a change because one of 6/21/2022 2:34:11 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -33.29577988475639 (T) = (0 -9.650188566950522) / Math.Sqrt((0.024671619462356832 / (20)) + (0.06198761371580186 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.22253132885400148 = (7.893612490075482 - 9.650188566950522) / 7.893612490075482 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.40714376038205 > 1.2134942724024946.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 7/13/2022 11:09:36 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.270103870937678 (T) = (0 -2.4170550111434776) / Math.Sqrt((0.08259538996092379 / (20)) + (0.002242994726372124 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -1.4759287198348576 = (0.9762215655807301 - 2.4170550111434776) / 0.9762215655807301 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.525799606342959 > 10.320412090356484.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -19.073749321988384 (T) = (0 -11.209132451982953) / Math.Sqrt((0.035570481359565115 / (20)) + (0.15537703765297123 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1468570197879631 = (9.77378370501264 - 11.209132451982953) / 9.77378370501264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_GC<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AllocateArray - Duration of single invocation 185.11 ns 207.89 ns 1.12 0.07 False
AllocateUninitializedArray - Duration of single invocation 518.71 ns 620.65 ns 1.20 0.02 False
AllocateUninitializedArray - Duration of single invocation 173.53 ns 198.69 ns 1.15 0.08 False
AllocateArray - Duration of single invocation 794.15 ns 889.73 ns 1.12 0.00 False
NewOperator_Array - Duration of single invocation 51.76 ns 54.57 ns 1.05 0.02 False
AllocateArray - Duration of single invocation 73.61 ns 80.20 ns 1.09 0.02 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 207.8855972055829 > 204.20203866407547.
IsChangePoint: Marked as a change because one of 7/2/2022 10:25:38 PM, 7/6/2022 10:15:52 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -10.741058241684973 (T) = (0 -206.8615319637983) / Math.Sqrt((11.021704051731282 / (20)) + (31.727521267456797 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.06405847661078597 = (194.4080485338445 - 206.8615319637983) / 194.4080485338445 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_GC&lt;Char&gt;.AllocateUninitializedArray(length: 10000, pinned: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 620.6536509701001 > 530.7128874370636.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -47.920432057977635 (T) = (0 -621.1998775387385) / Math.Sqrt((42.624799009928836 / (20)) + (122.08763947920167 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.21304713500725286 = (512.0987137363169 - 621.1998775387385) / 512.0987137363169 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_GC&lt;Char&gt;.AllocateUninitializedArray(length: 1000, pinned: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 198.694019762825 > 187.95738589994647.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 12:32:56 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -12.398405491519839 (T) = (0 -197.47850830630344) / Math.Sqrt((20.615569762092516 / (20)) + (25.44030120591381 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.0882057591829436 = (181.47166254162818 - 197.47850830630344) / 181.47166254162818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_GC&lt;Char&gt;.AllocateArray(length: 10000, pinned: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 889.7335066683434 > 836.935210164716.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -45.95590729451196 (T) = (0 -897.6400108346255) / Math.Sqrt((45.86927194712058 / (20)) + (83.74086606191021 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.12011115764412032 = (801.3847596363486 - 897.6400108346255) / 801.3847596363486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_GC&lt;Char&gt;.NewOperator_Array(length: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.57173106491075 > 53.81808527345724.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/29/2022 7:32:35 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -26.008411410294894 (T) = (0 -54.42672862432958) / Math.Sqrt((0.12866310872919393 / (20)) + (0.255184429643471 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.0571838741873713 = (51.48274576753825 - 54.42672862432958) / 51.48274576753825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_GC&lt;Char&gt;.AllocateArray(length: 1000, pinned: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.20051550926539 > 76.54091256324122.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/1/2022 7:34:46 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -6.96712506564 (T) = (0 -78.56460857430446) / Math.Sqrt((3.7630644930919064 / (20)) + (4.990680686795601 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.05219636775076797 = (74.66724936738608 - 78.56460857430446) / 74.66724936738608 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetChars - Duration of single invocation 29.57 ns 31.67 ns 1.07 0.01 False
GetBytes - Duration of single invocation 18.82 ns 21.72 ns 1.15 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.665358881874702 > 30.997898018101036.
IsChangePoint: Marked as a change because one of 6/17/2022 9:56:33 AM, 6/30/2022 12:21:07 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -16.905225403968334 (T) = (0 -31.517232307772144) / Math.Sqrt((0.11786017748583939 / (20)) + (0.20881218667472873 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.05993403459653955 = (29.735088485736828 - 31.517232307772144) / 29.735088485736828 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.723880508232607 > 19.704434470238624.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.84851139732314 (T) = (0 -20.936756331834804) / Math.Sqrt((0.039230635894794114 / (20)) + (0.23720475385831477 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.10735418762049165 = (18.907009668536308 - 20.936756331834804) / 18.907009668536308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 13.62 ns 14.85 ns 1.09 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.85483389724456 > 14.165511128170023.
IsChangePoint: Marked as a change because one of 6/17/2022 8:24:28 PM, 7/7/2022 8:07:51 AM, 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -15.73838461341618 (T) = (0 -15.034512700165148) / Math.Sqrt((0.08690732262957067 / (20)) + (0.0601938672978021 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.08703635598050703 = (13.830735851151928 - 15.034512700165148) / 13.830735851151928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Indexer_FirstElement_Array - Duration of single invocation 0.28 ns 1.39 ns 5.00 0.61 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.Indexer_FirstElement_Array

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.391974796254011 > 0.8192295669064288.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -17.53920623448209 (T) = (0 -1.392015416965496) / Math.Sqrt((0.032300726612420165 / (20)) + (4.573729130714679E-06 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -1.0258300646434415 = (0.6871333589426708 - 1.392015416965496) / 0.6871333589426708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse_SequentialAsync - Duration of single invocation 208.44 ns 222.66 ns 1.07 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 4096, Chunks: 1)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 222.65579260758815 > 221.457501639882.
IsChangePoint: Marked as a change because one of 7/23/2022 2:45:25 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -11.878068288821222 (T) = (0 -222.91675259084477) / Math.Sqrt((14.970014091549865 / (20)) + (38.66600911081634 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.07501884239822525 = (207.3607864337958 - 222.91675259084477) / 207.3607864337958 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in Benchstone.BenchI.NDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 336.74 ms 421.77 ms 1.25 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.NDhrystone*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.NDhrystone.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 421.76587333333333 > 342.11833250000007.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -75.81756606023059 (T) = (0 -425052158.1895604) / Math.Sqrt((29337210284390.195 / (20)) + (3781819682591.0635 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.2868148212533843 = (330313383.99999994 - 425052158.1895604) / 330313383.99999994 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultipleSerial - Duration of single invocation 398.34 ns 433.77 ns 1.09 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 433.77495000000005 > 423.348468173077.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -10.293846704155788 (T) = (0 -435.09047387240633) / Math.Sqrt((73.86745787724601 / (20)) + (124.6646976587069 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.06580342214001784 = (408.2276945581501 - 435.09047387240633) / 408.2276945581501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDocumentElement - Duration of single invocation 5.09 ns 6.39 ns 1.26 0.40 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.390965770929557 > 5.787276180742176.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -13.03618896132474 (T) = (0 -6.329505443248449) / Math.Sqrt((0.0605676095520368 / (20)) + (0.003795481689553026 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.13007462583501003 = (5.600962359960599 - 6.329505443248449) / 5.600962359960599 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Globalization.Tests.Perf_CultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCurrentCulture - Duration of single invocation 3.34 ns 5.16 ns 1.54 0.53 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.Perf_CultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_CultureInfo.GetCurrentCulture

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.1562629186897455 > 3.66760576648604.
IsChangePoint: Marked as a change because one of 6/28/2022 7:07:47 PM, 6/30/2022 12:21:07 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -14.114851470612285 (T) = (0 -4.847661389060954) / Math.Sqrt((0.10664711175812318 / (20)) + (0.05477676065620328 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.31296152024458407 = (3.692158006396037 - 4.847661389060954) / 3.692158006396037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.74 μs 4.02 μs 1.07 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.01681886479946 > 3.77550760186745.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -12.274076682081121 (T) = (0 -3906.5857902383054) / Math.Sqrt((2361.0528400457997 / (20)) + (19924.320501100116 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.08458683517607547 = (3601.9114961911714 - 3906.5857902383054) / 3601.9114961911714 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Collections.Tests.Add_Remove_SteadyState<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 7.41 ns 10.89 ns 1.47 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Add_Remove_SteadyState<String>.Queue(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.89151186884366 > 7.7993178233649605.
IsChangePoint: Marked as a change because one of 6/19/2022 4:44:31 AM, 6/19/2022 8:36:06 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -5.870168223765157 (T) = (0 -8.204036811326244) / Math.Sqrt((0.004011238714883192 / (20)) + (0.5716196682498695 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09425192954897661 = (7.497393049795894 - 8.204036811326244) / 7.497393049795894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetNames - Duration of single invocation 16.99 ns 21.69 ns 1.28 0.09 False
GetName - Duration of single invocation 15.02 ns 16.38 ns 1.09 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.GetNames

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.68527751223129 > 18.18181433295045.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/9/2022 7:28:50 AM, 7/16/2022 12:42:08 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.46923461193979 (T) = (0 -20.68021346282646) / Math.Sqrt((0.10076293813634228 / (18)) + (0.6328519523892567 / (40))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (40) - 2, .025) and -0.19917152203722027 = (17.245417425935653 - 20.68021346282646) / 17.245417425935653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.GetName

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.377183304902466 > 15.628688484273676.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 6/20/2022 2:50:37 PM, 6/21/2022 5:13:43 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -37.669113488727774 (T) = (0 -16.385406836816905) / Math.Sqrt((0.03536967010403228 / (20)) + (0.008373996991665708 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.11388408183637595 = (14.710154408350581 - 16.385406836816905) / 14.710154408350581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 8.61 ns 10.18 ns 1.18 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 255)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.175508345259244 > 9.06998365151907.
IsChangePoint: Marked as a change because one of 6/17/2022 8:24:28 PM, 6/30/2022 2:59:47 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -129.96897696161435 (T) = (0 -10.182010057675303) / Math.Sqrt((0.0020601779045921393 / (20)) + (0.0012576266260748959 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1737202615469397 = (8.674988744128536 - 10.182010057675303) / 8.674988744128536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in Benchstone.MDBenchI.MDNDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 342.37 ms 379.33 ms 1.11 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDNDhrystone*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDNDhrystone.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 379.3334 > 355.6554365.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 7/24/2022 8:36:55 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -33.19308470384586 (T) = (0 -380925557.9798535) / Math.Sqrt((24115146822564.98 / (20)) + (5307332621883.726 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.11211300159356284 = (342524147.66666675 - 380925557.9798535) / 342524147.66666675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 13.87 ns 14.96 ns 1.08 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 32767)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.957531892982963 > 14.52824694552963.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 8/1/2022 8:26:50 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -7.877312143759613 (T) = (0 -15.288158214103476) / Math.Sqrt((0.13323952548385573 / (20)) + (0.11908322204923803 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.053282555794231454 = (14.514773960701728 - 15.288158214103476) / 14.514773960701728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GenericClassWithSTringGenericInstanceMethod - Duration of single invocation 195.03 μs 250.80 μs 1.29 0.59 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.GenericClassWithSTringGenericInstanceMethod

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.80057397959186 > 203.19976821108222.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -16.29829274990816 (T) = (0 -250646.50985180997) / Math.Sqrt((377982178.69677466 / (20)) + (6811.135757484958 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.39408806508994415 = (179792.45079875115 - 250646.50985180997) / 179792.45079875115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Threading.Channels.Tests.UnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadAsyncThenWriteAsync - Duration of single invocation 98.13 ns 108.25 ns 1.10 0.14 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'

Payloads

Baseline
Compare

Histogram

System.Threading.Channels.Tests.UnboundedChannelPerfTests.ReadAsyncThenWriteAsync

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.25047470628286 > 102.91774498620242.
IsChangePoint: Marked as a change because one of 6/17/2022 11:02:28 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -9.178566375317349 (T) = (0 -104.94111912454426) / Math.Sqrt((14.231077129321262 / (20)) + (7.887446259610834 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09096168238780317 = (96.19138858741414 - 104.94111912454426) / 96.19138858741414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 24.60 μs 26.43 μs 1.07 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.428997564935063 > 25.85962442736917.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -36.80239921711956 (T) = (0 -26370.132810498104) / Math.Sqrt((3592.082047541768 / (20)) + (78895.43394099578 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.06922330844396156 = (24662.88622988822 - 26370.132810498104) / 24662.88622988822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Threading.Tests.Perf_Thread

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CurrentThread - Duration of single invocation 3.54 ns 4.81 ns 1.36 0.55 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Thread*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Thread.CurrentThread

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.808085789354252 > 3.6956737488488023.
IsChangePoint: Marked as a change because one of 6/22/2022 4:47:31 AM, 7/4/2022 6:56:50 PM, 7/5/2022 3:17:18 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -18.814134503433877 (T) = (0 -4.5459521402089935) / Math.Sqrt((0.054281503606134394 / (20)) + (0.04550916585487427 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.3456361291184121 = (3.3782922751838234 - 4.5459521402089935) / 3.3782922751838234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 0.28 ns 1.75 ns 6.28 0.64 False
ToString - Duration of single invocation 0.28 ns 1.42 ns 5.09 0.67 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.ToString(address: 1020:3040:5060:7080:9010:1112:1314:1516)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7487302261639275 > 0.8527660376431528.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -16.6660198668279 (T) = (0 -1.7494017914838125) / Math.Sqrt((0.09177703630941043 / (20)) + (0.00016619560428432828 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -1.8219941369968555 = (0.6199168767039015 - 1.7494017914838125) / 0.6199168767039015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.ToString(address: 143.24.20.36)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4235683293535861 > 0.30645089876909876.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -17.616207477577547 (T) = (0 -1.437056401530959) / Math.Sqrt((0.056117368464182714 / (20)) + (0.002326028077421749 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -1.9072667955536506 = (0.4942980822154957 - 1.437056401530959) / 0.4942980822154957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Threading.Tests.Perf_ThreadStatic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SetThreadStatic - Duration of single invocation 5.01 ns 6.13 ns 1.22 0.37 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_ThreadStatic*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_ThreadStatic.SetThreadStatic

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.125306997607445 > 4.942753834451808.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.452801004058227 (T) = (0 -5.890646221263502) / Math.Sqrt((0.03427780988078299 / (20)) + (0.029916237253784745 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.23322557836300564 = (4.776616966607842 - 5.890646221263502) / 4.776616966607842 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor4 - Duration of single invocation 4.28 ns 5.69 ns 1.33 0.09 False
Ctor3 - Duration of single invocation 4.03 ns 5.54 ns 1.37 0.09 False
Ctor2 - Duration of single invocation 3.77 ns 4.88 ns 1.30 0.12 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.Ctor4

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.6901525214640545 > 4.5682503995978125.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -56.78915850004387 (T) = (0 -5.702015760839844) / Math.Sqrt((0.007504457725522672 / (20)) + (0.003784524984746471 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.27531319251421243 = (4.471070945011258 - 5.702015760839844) / 4.471070945011258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.Ctor3

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.538531155192904 > 4.231858692003181.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -35.1902444022604 (T) = (0 -5.599613618429966) / Math.Sqrt((0.021539768540646313 / (20)) + (0.014225825899175667 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.3121047401714974 = (4.267657487235412 - 5.599613618429966) / 4.267657487235412 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.Ctor2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.883110430124897 > 3.9618521382085063.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -15.829696454324058 (T) = (0 -4.887110853780885) / Math.Sqrt((0.04218426964416101 / (17)) + (0.0017191524645046197 / (40))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (40) - 2, .025) and -0.19437517875369187 = (4.091771949648597 - 4.887110853780885) / 4.091771949648597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.IO.Pipelines.Tests.Perf_Pipe

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadAsync - Duration of single invocation 2.40 ms 2.72 ms 1.13 0.08 False
ReadAsyncWithCancellationToken - Duration of single invocation 2.81 ms 3.12 ms 1.11 0.07 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Pipelines.Tests.Perf_Pipe*'

Payloads

Baseline
Compare

Histogram

System.IO.Pipelines.Tests.Perf_Pipe.ReadAsync

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.721407185185185 > 2.5370649113144137.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -20.30498482882078 (T) = (0 -2655517.106175881) / Math.Sqrt((850099905.710045 / (20)) + (5006560733.25485 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.10989048746228322 = (2392593.806482301 - 2655517.106175881) / 2392593.806482301 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Pipelines.Tests.Perf_Pipe.ReadAsyncWithCancellationToken

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.117321785714286 > 2.9872368257392403.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -11.249647265230143 (T) = (0 -3131863.4728492955) / Math.Sqrt((8687725394.44361 / (20)) + (5107262410.930206 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09308609642826367 = (2865157.175709106 - 3131863.4728492955) / 2865157.175709106 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Reflection.Activator<EmptyStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateInstanceType - Duration of single invocation 7.86 ns 9.36 ns 1.19 0.51 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Activator&lt;EmptyStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Activator<EmptyStruct>.CreateInstanceType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.364618840199073 > 8.625489476420134.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -6.365496309823058 (T) = (0 -9.715273591666156) / Math.Sqrt((0.28886323961957244 / (7)) + (0.2581963275398308 / (40))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (40) - 2, .025) and -0.1670389459145865 = (8.324720974973529 - 9.715273591666156) / 8.324720974973529 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Transient - Duration of single invocation 29.16 ns 31.55 ns 1.08 0.08 False
NoDI - Duration of single invocation 2.03 ns 3.47 ns 1.71 0.38 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.Transient

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.545287347055268 > 30.26939639812723.
IsChangePoint: Marked as a change because one of 6/22/2022 6:53:40 PM, 7/6/2022 7:05:00 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -11.683872744578707 (T) = (0 -30.576666685344282) / Math.Sqrt((0.20304976957096116 / (20)) + (0.3351085947317474 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.054870100345816394 = (28.98619145174404 - 30.576666685344282) / 28.98619145174404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.DependencyInjection.GetService.NoDI

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4691192053609634 > 2.3790039715342366.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -44.3283642135607 (T) = (0 -3.41210687315371) / Math.Sqrt((0.008919646830036957 / (20)) + (0.010334565761328986 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.5262237835496795 = (2.2356530607967975 - 3.41210687315371) / 2.2356530607967975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Xml.Linq.Perf_XDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Create - Duration of single invocation 2.75 ns 4.33 ns 1.57 0.18 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XDocument*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XDocument.Create

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.329781271620684 > 3.2303309966261855.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.686603330189477 (T) = (0 -4.294230213262024) / Math.Sqrt((0.018529997625444383 / (7)) + (0.011437756584398244 / (40))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (40) - 2, .025) and -0.42598576209348066 = (3.0114117036888866 - 4.294230213262024) / 3.0114117036888866 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToByteArray - Duration of single invocation 14.68 ns 16.47 ns 1.12 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: -2147483648)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.470923478610306 > 16.179184190348558.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 7/28/2022 3:42:13 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -19.437928979118833 (T) = (0 -16.885826393714982) / Math.Sqrt((0.07523083251427096 / (20)) + (0.07024608364904326 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.09391973434251907 = (15.436074387911017 - 16.885826393714982) / 15.436074387911017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Collections.Tests.Add_Remove_SteadyState<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableStack - Duration of single invocation 8.37 ns 9.47 ns 1.13 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Add_Remove_SteadyState<Int32>.ImmutableStack(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.473323456990505 > 8.762060331664442.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -12.711969465662323 (T) = (0 -9.12613821954822) / Math.Sqrt((0.06257806638724496 / (20)) + (0.033359643632765945 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.09639821176727607 = (8.323744166672679 - 9.12613821954822) / 8.323744166672679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 2.67 ns 4.05 ns 1.52 0.38 False
ToString - Duration of single invocation 13.82 ns 15.45 ns 1.12 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.054118544180104 > 2.9367046379963604.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -47.36300791943082 (T) = (0 -4.054908978098184) / Math.Sqrt((0.015486767223002325 / (20)) + (0.0001716940722589875 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.48352477982605 = (2.733293729393344 - 4.054908978098184) / 2.733293729393344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.453654094662653 > 14.22535997836031.
IsChangePoint: Marked as a change because one of 6/22/2022 10:05:16 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -44.53435993713057 (T) = (0 -15.41828919845943) / Math.Sqrt((0.03766656628510676 / (20)) + (0.006226909397922364 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1499898057177178 = (13.407326849159983 - 15.41828919845943) / 13.407326849159983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToArray - Duration of single invocation 28.68 ns 31.48 ns 1.10 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.ToArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.47633928768478 > 28.08555970096814.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -20.28498192160749 (T) = (0 -31.417758307735504) / Math.Sqrt((1.467084723747578 / (20)) + (0.17351737140087253 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.21965168393845194 = (25.759615406164567 - 31.417758307735504) / 25.759615406164567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 11.07 ns 12.85 ns 1.16 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: 127)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.84874363724228 > 11.585475116894216.
IsChangePoint: Marked as a change because one of 6/21/2022 5:13:43 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -51.64598742119865 (T) = (0 -12.887752396111008) / Math.Sqrt((0.019460426517360015 / (20)) + (0.027349669211080386 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.19490500015353462 = (10.785587468840658 - 12.887752396111008) / 10.785587468840658 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@dakersnar dakersnar transferred this issue from dotnet/perf-autofiling-issues Oct 14, 2022
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dakersnar dakersnar added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark labels Oct 14, 2022
@dakersnar dakersnar changed the title [Perf] Windows/x64: 129 Regressions on 8/9/2022 4:14:59 PM Regressions in Perf_String, Perf_Stringbuilder, other on 8/9/2022 Oct 14, 2022
@dakersnar
Copy link
Contributor

dakersnar commented Oct 14, 2022 •
edited
Loading

Working on the 7.0-rc2 vs 6.0 perf report, we discovered evidence that a number of these benchmarks had regressed on August 9th. While investigating, we uncovered this issue that had not been triaged yet.

Here is the data from the report for select benchmarks:

System.Tests.Perf_String.CtorCharCount(size: 1)

Result Ratio Alloc Delta Operating System     Bit   Processor Name                                 Modality
Same   0.98            +0 ubuntu 18.04         Arm64 Unknown processor                                    
Same   0.96            +0 Windows 11           Arm64 Unknown processor                                    
Noise   -              +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Noise   -              +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Same   0.92            +0 macOS Monterey 12.6 Arm64 Apple M1                                              
Same   0.90            +0 macOS Monterey 12.6 Arm64 Apple M1 Max                                          
Slower 0.46            +0 Windows 10           X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)        
Slower 0.69            +0 Windows 11           X64   AMD Ryzen Threadripper PRO 3945WX 12-Cores            
Slower 0.64            +0 Windows 11           X64   AMD Ryzen 9 5900X                             several?
Noise   -              +0 Windows 11           X64   AMD Ryzen 9 7950X                                    
Slower 0.59            +0 Windows 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Slower 0.76            +0 debian 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Slower 0.82            +0 ubuntu 18.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.66            +0 ubuntu 18.04         X64   Intel Xeon CPU E5-1650 v4 3.60GHz                    
Slower 0.71            +0 ubuntu 20.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.59            +0 ubuntu 20.04         X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) bimodal
Slower 0.81            +0 ubuntu 20.04         X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Slower 0.83            +0 macOS Big Sur 11.7   X64   Intel Core i5-4278U CPU 2.60GHz (Haswell)            
Slower 0.79            +0 macOS Monterey 12.6 X64   Intel Core i7-4870HQ CPU 2.50GHz (Haswell)            

System.Tests.Perf_String.CtorCharCount(size: 10)

Result Ratio Alloc Delta Operating System     Bit   Processor Name                                 Modality
Faster 1.15            +0 ubuntu 18.04         Arm64 Unknown processor                                    
Same   0.99            +0 Windows 11           Arm64 Unknown processor                                    
Same   1.11            +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Same   1.06            +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Noise   -              +0 macOS Monterey 12.6 Arm64 Apple M1                                              
Noise   -              +0 macOS Monterey 12.6 Arm64 Apple M1 Max                                          
Slower 0.52            +0 Windows 10           X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) bimodal
Noise   -              +0 Windows 11           X64   AMD Ryzen Threadripper PRO 3945WX 12-Cores            
Slower 0.62            +0 Windows 11           X64   AMD Ryzen 9 5900X                                    
Noise   -              +0 Windows 11           X64   AMD Ryzen 9 7950X                             bimodal
Slower 0.58            +0 Windows 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Slower 0.78            +0 debian 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Noise   -              +0 ubuntu 18.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.81            +0 ubuntu 18.04         X64   Intel Xeon CPU E5-1650 v4 3.60GHz                    
Slower 0.67            +0 ubuntu 20.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.64            +0 ubuntu 20.04         X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)        
Slower 0.79            +0 ubuntu 20.04         X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same   0.93            +0 macOS Big Sur 11.7   X64   Intel Core i5-4278U CPU 2.60GHz (Haswell)            
Same   0.91            +0 macOS Monterey 12.6 X64   Intel Core i7-4870HQ CPU 2.50GHz (Haswell)            

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

Result Ratio Alloc Delta Operating System     Bit   Processor Name                                 Modality
Same   1.09            +0 ubuntu 18.04         Arm64 Unknown processor                                    
Same   1.07            +0 Windows 11           Arm64 Unknown processor                                    
Same   1.04            +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Same   0.97            +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Noise   -              +0 macOS Monterey 12.6 Arm64 Apple M1                                              
Noise   -              +0 macOS Monterey 12.6 Arm64 Apple M1 Max                                          
Slower 0.61            +0 Windows 10           X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) several?
Slower 0.77            +0 Windows 11           X64   AMD Ryzen Threadripper PRO 3945WX 12-Cores            
Slower 0.68            +0 Windows 11           X64   AMD Ryzen 9 5900X                                    
Same   0.94            +0 Windows 11           X64   AMD Ryzen 9 7950X                             several?
Slower 0.75            +0 Windows 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Slower 0.85            +0 debian 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same   0.96            +0 ubuntu 18.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.79            +0 ubuntu 18.04         X64   Intel Xeon CPU E5-1650 v4 3.60GHz                    
Slower 0.83            +0 ubuntu 20.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.88            +0 ubuntu 20.04         X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)        
Slower 0.88            +0 ubuntu 20.04         X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same   0.96            +0 macOS Big Sur 11.7   X64   Intel Core i5-4278U CPU 2.60GHz (Haswell)            
Same   0.95            +0 macOS Monterey 12.6 X64   Intel Core i7-4870HQ CPU 2.50GHz (Haswell)            

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000)

Result Ratio Alloc Delta Operating System     Bit   Processor Name                                 Modality
Same    0.91          +0 ubuntu 18.04         Arm64 Unknown processor                                    
Same    0.96          -2 Windows 11           Arm64 Unknown processor                                    
Slower  0.78          -1 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Slower  0.77          +1 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Slower  0.85          +0 macOS Monterey 12.6 Arm64 Apple M1                                              
Same    0.94         +76 macOS Monterey 12.6 Arm64 Apple M1 Max                                          
Slower  0.60          +5 Windows 10           X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)        
Slower  0.85         -20 Windows 11           X64   AMD Ryzen Threadripper PRO 3945WX 12-Cores            
Slower  0.78          +0 Windows 11           X64   AMD Ryzen 9 5900X                                    
Same    0.89         +17 Windows 11           X64   AMD Ryzen 9 7950X                                    
Slower  0.80          +0 Windows 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same    0.93          +0 debian 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same    0.95         +18 ubuntu 18.04         X64   AMD Ryzen 9 5900X                                    
Same    0.90         +15 ubuntu 18.04         X64   Intel Xeon CPU E5-1650 v4 3.60GHz                    
Slower  0.88         +17 ubuntu 20.04         X64   AMD Ryzen 9 5900X                                    
Slower  0.58         +21 ubuntu 20.04         X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) bimodal
Slower  0.89          -1 ubuntu 20.04         X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Slower  0.89          -6 macOS Big Sur 11.7   X64   Intel Core i5-4278U CPU 2.60GHz (Haswell)            
Slower  0.84          -4 macOS Monterey 12.6 X64   Intel Core i7-4870HQ CPU 2.50GHz (Haswell)            

System.Tests.Perf_String.CtorCharCount(size: 100)

Notably we observed an improvement on this version of the benchmark, indicating that the above regressions could be the result of an intentional tradeoff.

Result Ratio Alloc Delta Operating System     Bit   Processor Name                                 Modality
Faster  1.19          +0 ubuntu 18.04         Arm64 Unknown processor                                    
Faster  1.36          +0 Windows 11           Arm64 Unknown processor                                    
Faster  1.42          +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Faster  1.46          +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Faster  1.58          +0 macOS Monterey 12.6 Arm64 Apple M1                                              
Faster  1.60          +0 macOS Monterey 12.6 Arm64 Apple M1 Max                                          
Same    0.96          +0 Windows 10           X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) several?
Faster  1.41          +0 Windows 11           X64   AMD Ryzen Threadripper PRO 3945WX 12-Cores            
Same    1.12          +0 Windows 11           X64   AMD Ryzen 9 5900X                                    
Faster  1.32          +0 Windows 11           X64   AMD Ryzen 9 7950X                                    
Faster  1.16          +0 Windows 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Faster  1.21          +0 debian 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same    1.11          +0 ubuntu 18.04         X64   AMD Ryzen 9 5900X                                    
Faster  1.28          +0 ubuntu 18.04         X64   Intel Xeon CPU E5-1650 v4 3.60GHz                    
Same    0.90          +0 ubuntu 20.04         X64   AMD Ryzen 9 5900X                                    
Slower  0.82          +0 ubuntu 20.04         X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)        
Faster  1.18          +0 ubuntu 20.04         X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Faster  1.37          +0 macOS Big Sur 11.7   X64   Intel Core i5-4278U CPU 2.60GHz (Haswell)            
Faster  1.45          +0 macOS Monterey 12.6 X64   Intel Core i7-4870HQ CPU 2.50GHz (Haswell)            

@ghost
Copy link

ghost commented Oct 14, 2022

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Remove_IntInt - Duration of single invocation 11.88 ns 13.46 ns 1.13 0.04 False
Trim_CharArr - Duration of single invocation 15.72 ns 17.44 ns 1.11 0.03 False
Insert - Duration of single invocation 14.84 ns 16.75 ns 1.13 0.03 False
Substring_Int - Duration of single invocation 7.71 ns 9.22 ns 1.20 0.10 False
CtorCharCount - Duration of single invocation 9.63 ns 11.28 ns 1.17 0.06 False
Trim_CharArr - Duration of single invocation 15.14 ns 16.58 ns 1.09 0.03 False
TrimStart_CharArr - Duration of single invocation 13.07 ns 15.04 ns 1.15 0.04 False
Trim_CharArr - Duration of single invocation 18.13 ns 19.37 ns 1.07 0.04 False
GetChars - Duration of single invocation 16.15 ns 18.18 ns 1.13 0.04 False
Replace_Char - Duration of single invocation 25.73 ns 27.68 ns 1.08 0.04 False
Substring_IntInt - Duration of single invocation 7.61 ns 9.43 ns 1.24 0.09 False
Insert - Duration of single invocation 15.24 ns 17.36 ns 1.14 0.03 False
PadLeft - Duration of single invocation 12.05 ns 13.09 ns 1.09 0.15 False
Replace_Char - Duration of single invocation 21.17 ns 23.10 ns 1.09 0.02 False
Substring_IntInt - Duration of single invocation 7.83 ns 9.84 ns 1.26 0.07 False
Remove_IntInt - Duration of single invocation 12.06 ns 13.85 ns 1.15 0.04 False
CtorCharCount - Duration of single invocation 7.96 ns 9.85 ns 1.24 0.08 False
TrimEnd_CharArr - Duration of single invocation 14.36 ns 15.62 ns 1.09 0.05 False
Substring_Int - Duration of single invocation 7.76 ns 9.15 ns 1.18 0.12 False
Replace_String - Duration of single invocation 28.54 ns 30.04 ns 1.05 0.03 False
Substring_IntInt - Duration of single invocation 8.39 ns 10.07 ns 1.20 0.09 False
Remove_IntInt - Duration of single invocation 12.31 ns 13.72 ns 1.11 0.06 False
CtorCharCount - Duration of single invocation 16.91 ns 18.82 ns 1.11 0.06 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.458880221615381 > 12.323685492321225.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -63.05248742051773 (T) = (0 -13.35949958004901) / Math.Sqrt((0.008845259925720085 / (20)) + (0.011335104513203001 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.14565549198003522 = (11.661009503790533 - 13.35949958004901) / 11.661009503790533 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.442991300883428 > 16.389468749734654.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 3:55:24 AM, 7/26/2022 5:16:37 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -17.53574088487783 (T) = (0 -17.57255894206272) / Math.Sqrt((0.13017317560899197 / (20)) + (0.028604759004090628 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09267493259124028 = (16.082147048427306 - 17.57255894206272) / 16.082147048427306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.745340494409 > 15.7702016065585.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/4/2022 6:29:12 PM, 7/6/2022 1:39:15 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -8.09343774740642 (T) = (0 -16.688767663312245) / Math.Sqrt((0.2188977269711232 / (20)) + (0.006546919949689709 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.05386708472754068 = (15.835742386457435 - 16.688767663312245) / 15.835742386457435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 7)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.223322921851516 > 7.792330455508882.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -36.70253957161099 (T) = (0 -9.672184263775653) / Math.Sqrt((0.008598152621522555 / (20)) + (0.12233827470101226 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.2888607042548379 = (7.50444499692283 - 9.672184263775653) / 7.50444499692283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.28234880445019 > 10.29204619995384.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -40.20782312602304 (T) = (0 -11.373944422243357) / Math.Sqrt((0.02384996979671384 / (20)) + (0.002811354960092524 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.14366937585733444 = (9.945133324670035 - 11.373944422243357) / 9.945133324670035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.575885270328836 > 15.516340913481352.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 1:39:15 AM, 7/28/2022 3:42:13 AM, 8/1/2022 6:03:22 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -9.45104317668793 (T) = (0 -16.94481879422906) / Math.Sqrt((0.3427480328011729 / (20)) + (0.08191045734979938 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.08384689591350462 = (15.633959794614134 - 16.94481879422906) / 15.633959794614134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.035952923753275 > 13.698417077458656.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 1:39:15 AM, 7/24/2022 8:36:55 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -11.682605554450639 (T) = (0 -15.013609700812694) / Math.Sqrt((0.22908401492976385 / (20)) + (0.0145648323958909 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09240981170912917 = (13.743569070771306 - 15.013609700812694) / 13.743569070771306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: " Te st  ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.37070633908783 > 18.96510022255503.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 6/30/2022 4:27:12 PM, 7/7/2022 8:07:51 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -29.72842177195503 (T) = (0 -19.62300556564896) / Math.Sqrt((0.039357702102435256 / (18)) + (0.039999662268168736 / (40))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (40) - 2, .025) and -0.09351708092949 = (17.944855099080296 - 19.62300556564896) / 17.944855099080296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.GetChars(size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.184019786236682 > 17.103696802551763.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -53.1012790865293 (T) = (0 -18.27982243488964) / Math.Sqrt((0.025615712287388893 / (20)) + (0.016321472072452943 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.13556125049336984 = (16.097610258317253 - 18.27982243488964) / 16.097610258317253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.67953214612285 > 26.211930390726824.
IsChangePoint: Marked as a change because one of 7/7/2022 8:07:51 AM, 7/10/2022 1:04:01 AM, 7/13/2022 1:49:45 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.37369442086753 (T) = (0 -27.52857211363341) / Math.Sqrt((0.3986877818782333 / (20)) + (0.04998724862440364 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.13416350297379895 = (24.27213716668977 - 27.52857211363341) / 24.27213716668977 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.426734291823244 > 7.9117523499889435.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 12:32:56 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -40.449662534535655 (T) = (0 -9.37004646612724) / Math.Sqrt((0.054833057708211695 / (20)) + (0.01400185947370848 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.3158474071842175 = (7.120921783915817 - 9.37004646612724) / 7.120921783915817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.360342888810553 > 16.63104364207198.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -15.354982829432739 (T) = (0 -17.289014424679973) / Math.Sqrt((0.11169264642952915 / (18)) + (0.008356415597080974 / (40))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (40) - 2, .025) and -0.07657553246351055 = (16.059267467390605 - 17.289014424679973) / 16.059267467390605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.PadLeft(n: 18)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.094671984522298 > 11.998458424158024.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -16.814737755413415 (T) = (0 -13.781697574207982) / Math.Sqrt((0.11980002360747036 / (20)) + (0.37176288283189796 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1776297510001823 = (11.702912195028137 - 13.781697574207982) / 11.702912195028137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.098672440738362 > 21.90359221421459.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -10.17941463137221 (T) = (0 -23.075328685809435) / Math.Sqrt((0.42473609067567436 / (7)) + (0.017770962796527304 / (40))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (40) - 2, .025) and -0.12241134795814908 = (20.55870936068782 - 23.075328685809435) / 20.55870936068782 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.843047748606525 > 8.205400107385309.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -24.88788601176509 (T) = (0 -9.922853850421653) / Math.Sqrt((0.08297911298044891 / (20)) + (0.2832283316313345 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.36200378167964936 = (7.285481864216704 - 9.922853850421653) / 7.285481864216704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.847245706923978 > 12.784272639740891.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -39.55151525449711 (T) = (0 -13.956003830970548) / Math.Sqrt((0.007689134819444583 / (7)) + (0.03611117269714392 / (40))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (40) - 2, .025) and -0.14518658544241028 = (12.186663735306542 - 13.956003830970548) / 12.186663735306542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 1)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.850974994977735 > 8.372996448790467.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -31.99966318274964 (T) = (0 -9.893426406232269) / Math.Sqrt((0.019631505017476288 / (20)) + (0.08239181122689478 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.21709986940849477 = (8.128689070552962 - 9.893426406232269) / 8.128689070552962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.616962333457307 > 15.08212603434616.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 6/30/2022 4:27:12 PM, 7/6/2022 3:55:24 AM, 7/26/2022 6:20:16 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.143272393779462 (T) = (0 -15.666648509497207) / Math.Sqrt((0.0531031761327196 / (17)) + (0.021396580008356952 / (40))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (40) - 2, .025) and -0.0934840325694977 = (14.327276890073376 - 15.666648509497207) / 14.327276890073376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.14664838078872 > 7.866964388609689.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -34.8772616286586 (T) = (0 -9.66122102625102) / Math.Sqrt((0.016226408411900234 / (20)) + (0.11677215103087843 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.2828673858697124 = (7.530958486173731 - 9.66122102625102) / 7.530958486173731 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.0424995318874 > 29.45401414871855.
IsChangePoint: Marked as a change because one of 6/17/2022 12:17:51 PM, 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -22.345187203220288 (T) = (0 -30.09628951628173) / Math.Sqrt((0.08341998465796059 / (20)) + (0.2692124876551383 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.08403363967062433 = (27.76324314568897 - 30.09628951628173) / 27.76324314568897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.065460869195261 > 8.863186282537988.
IsChangePoint: Marked as a change because one of 6/30/2022 4:27:12 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -34.09494349666231 (T) = (0 -10.47691481750297) / Math.Sqrt((0.029679385710329825 / (20)) + (0.10637482543009164 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.26498185835553084 = (8.282264878583238 - 10.47691481750297) / 8.282264878583238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.72135757462028 > 12.897907381147377.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -13.106291726793815 (T) = (0 -13.895130048375464) / Math.Sqrt((0.054250772856808134 / (20)) + (0.2615186672720924 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09977046482956478 = (12.634572842914853 - 13.895130048375464) / 12.634572842914853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.81634604031629 > 17.708728079417345.
IsChangePoint: Marked as a change because one of 6/21/2022 5:13:43 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -23.779207881011875 (T) = (0 -18.79626944340756) / Math.Sqrt((0.2756523566834548 / (20)) + (0.011766423251870505 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.1766074178431023 = (15.974971055225827 - 18.79626944340756) / 15.974971055225827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 543bcc5ee7d6a2b9471b016770227421c43a756e
Compare 4211dc40d06195682d4b0fc9a220bdb75732d720
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringHex - Duration of single invocation 19.28 ns 20.34 ns 1.06 0.06 False
ToString - Duration of single invocation 11.09 ns 12.27 ns 1.11 0.07 False
ToStringHex - Duration of single invocation 25.42 ns 27.64 ns 1.09 0.03 False
ToString - Duration of single invocation 16.97 ns 18.27 ns 1.08 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToStringHex(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.3438823277552 > 19.35999656916104.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -19.662082462612894 (T) = (0 -20.399557788406188) / Math.Sqrt((0.11301839374618175 / (20)) + (0.1023318492346544 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.09568535108814727 = (18.618080243700415 - 20.399557788406188) / 18.618080243700415 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.265364745407124 > 11.578409535243326.
IsChangePoint: Marked as a change because one of 6/27/2022 8:54:23 AM, 7/2/2022 3:52:19 AM, 7/4/2022 6:29:12 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -32.95671667775704 (T) = (0 -12.425452634575908) / Math.Sqrt((0.009606591167517048 / (20)) + (0.04130679073919822 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.11615651328518213 = (11.132356875295283 - 12.425452634575908) / 11.132356875295283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.638065026688533 > 25.665333977691123.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/6/2022 3:55:24 AM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -26.422305010768586 (T) = (0 -27.39829691503944) / Math.Sqrt((0.15747210833792727 / (20)) + (0.07024077056765961 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.10452691086761844 = (24.80545892133834 - 27.39829691503944) / 24.80545892133834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 2147483647)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.26866092072727 > 17.87855185425306.
IsChangePoint: Marked as a change because one of 7/2/2022 3:52:19 AM, 7/4/2022 6:29:12 PM, 8/5/2022 10:47:59 PM, 8/15/2022 12:02:22 PM falls between 8/5/2022 10:47:59 PM and 8/15/2022 12:02:22 PM.
IsRegressionStdDev: Marked as regression because -29.981702514702484 (T) = (0 -18.34288502955733) / Math.Sqrt((0.017645643245940756 / (20)) + (0.025699448514064045 / (40))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (40) - 2, .025) and -0.06817631586004307 = (17.17215103649676 - 18.34288502955733) / 17.17215103649676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: kunalspathak
Labels:

area-System.Runtime, tenet-performance, tenet-performance-benchmarks, untriaged, refs/heads/main, RunKind=micro, Windows 10.0.19042, Regression, CoreClr, x64
Milestone: -

@dakersnar
Copy link
Contributor

cc @stephentoub as this is string related, feel free to ping others who might be interested

@dakersnar
Copy link
Contributor

dakersnar commented Oct 14, 2022 •
edited
Loading

We also spotted a related regression for System.Tests.Perf_String.Trim(s: " Test") in the report. Same timing as the rest of these.

https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2frefs%2fheads%2fmain_x64_Windows 10.0.18362%2fSystem.Tests.Perf_String.Trim(s%3a " Test").html

04c610f6-9f50-4bb1-bee2-0e81ebf2276a

System.Tests.Perf_String.Trim(s: " Test")

Result Ratio Alloc Delta Operating System     Bit   Processor Name                                 Modality
Same   1.08            +0 ubuntu 18.04         Arm64 Unknown processor                                    
Slower 0.83            +0 Windows 11           Arm64 Unknown processor                                    
Noise   -              +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Slower 0.87            +0 Windows 11           Arm64 Microsoft SQ1 3.0 GHz                                
Noise   -              +0 macOS Monterey 12.6 Arm64 Apple M1                                              
Noise   -              +0 macOS Monterey 12.6 Arm64 Apple M1 Max                                          
Slower 0.67            +0 Windows 10           X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)        
Slower 0.85            +0 Windows 11           X64   AMD Ryzen Threadripper PRO 3945WX 12-Cores            
Noise   -              +0 Windows 11           X64   AMD Ryzen 9 5900X                             several?
Noise   -              +0 Windows 11           X64   AMD Ryzen 9 7950X                                    
Slower 0.76            +0 Windows 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Slower 0.82            +0 debian 11           X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same   0.95            +0 ubuntu 18.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.84            +0 ubuntu 18.04         X64   Intel Xeon CPU E5-1650 v4 3.60GHz                    
Slower 0.85            +0 ubuntu 20.04         X64   AMD Ryzen 9 5900X                                    
Slower 0.66            +0 ubuntu 20.04         X64   Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)        
Slower 0.87            +0 ubuntu 20.04         X64   Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)          
Same   0.98            +0 macOS Big Sur 11.7   X64   Intel Core i5-4278U CPU 2.60GHz (Haswell)            
Same   0.98            +0 macOS Monterey 12.6 X64   Intel Core i7-4870HQ CPU 2.50GHz (Haswell)            

@jozkee jozkee mentioned this issue Oct 18, 2022
Open
26 tasks
@dakersnar
Copy link
Contributor

After some investigation with @mrsharm, I think this is explained/tracked by #73592. This is considered to be "by design". Closing this as a duplicate.

@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Oct 24, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Nov 24, 2022
@jeffhandley jeffhandley added runtime-coreclr specific to the CoreCLR runtime and removed CoreClr labels Dec 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@kunalspathak kunalspathak

Labels
arch-x64 area-System.Runtime runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jeffhandley @kunalspathak @dakersnar