Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 92 Regressions on 1/14/2023 12:48:25 AM #11779

Closed
performanceautofiler bot opened this issue Jan 17, 2023 · 7 comments
Closed

[Perf] Linux/x64: 92 Regressions on 1/14/2023 12:48:25 AM #11779

performanceautofiler bot opened this issue Jan 17, 2023 · 7 comments
Assignees
@naricc
Labels
arch-x64 branch-refs/heads/main kind-micro_mono LinkedFromMaster Is this issue linked from the master perf issue mono-aot os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 247.87 ns 542.55 ns 2.19 0.01 False
ToString - Duration of single invocation 409.40 ns 717.57 ns 1.75 0.01 False
ToString - Duration of single invocation 406.28 ns 712.88 ns 1.75 0.01 False
Parse - Duration of single invocation 229.42 ns 291.86 ns 1.27 0.01 False
ToStringWithFormat - Duration of single invocation 412.50 ns 720.01 ns 1.75 0.01 False
ToStringWithFormat - Duration of single invocation 410.27 ns 724.33 ns 1.77 0.00 False
ToStringWithCultureInfo - Duration of single invocation 242.99 ns 529.40 ns 2.18 0.01 False
ToStringWithFormat - Duration of single invocation 419.64 ns 723.73 ns 1.72 0.01 False
ToStringWithFormat - Duration of single invocation 368.02 ns 661.46 ns 1.80 0.00 False
ToStringWithFormat - Duration of single invocation 278.05 ns 568.23 ns 2.04 0.01 False
Parse - Duration of single invocation 232.21 ns 293.55 ns 1.26 0.02 False
ToStringWithCultureInfo - Duration of single invocation 404.44 ns 708.01 ns 1.75 0.01 False
ToStringWithFormat - Duration of single invocation 562.15 ns 827.02 ns 1.47 0.05 False
TryParse - Duration of single invocation 232.29 ns 293.48 ns 1.26 0.01 False
ToString - Duration of single invocation 248.69 ns 540.81 ns 2.17 0.01 False
ToStringWithFormat - Duration of single invocation 280.57 ns 570.93 ns 2.03 0.00 False
TryParse - Duration of single invocation 228.05 ns 290.31 ns 1.27 0.01 False
ToStringWithFormat - Duration of single invocation 249.15 ns 546.86 ns 2.19 0.02 False
ToStringWithFormat - Duration of single invocation 415.66 ns 718.16 ns 1.73 0.00 False
ToStringWithFormat - Duration of single invocation 370.20 ns 658.43 ns 1.78 0.01 False
ToStringWithCultureInfo - Duration of single invocation 403.42 ns 719.57 ns 1.78 0.01 False
ToStringWithFormat - Duration of single invocation 278.06 ns 629.08 ns 2.26 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 542.5488570813854 > 261.47595709359217.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -265.4548397172526 (T) = (0 -547.0433976866557) / Math.Sqrt((3.5809396978685815 / (30)) + (21.53059545230805 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.1885864990043962 = (249.95283391152677 - 547.0433976866557) / 249.95283391152677 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 717.5700530694868 > 429.4393538116276.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -241.40521088999193 (T) = (0 -713.2190035279706) / Math.Sqrt((6.807931829468236 / (30)) + (25.56209342708702 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.7373667629855151 = (410.5172371908193 - 713.2190035279706) / 410.5172371908193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 712.8778521201872 > 426.7744941659534.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -164.52949458277033 (T) = (0 -713.4287537937576) / Math.Sqrt((10.549511236010556 / (29)) + (58.078245313806015 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7437405772855973 = (409.1369800571597 - 713.4287537937576) / 409.1369800571597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 291.8639129041514 > 240.73881707585858.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -79.79382481687524 (T) = (0 -291.63428299585695) / Math.Sqrt((4.7483071218865485 / (30)) + (8.73824158401557 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.2740872888149045 = (228.89662706479186 - 291.63428299585695) / 228.89662706479186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 720.0062404144174 > 434.6770993267689.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -173.24484354526302 (T) = (0 -718.7651259713097) / Math.Sqrt((6.457126330408947 / (29)) + (54.236011610861475 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7325462890173843 = (414.86056131808067 - 718.7651259713097) / 414.86056131808067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 724.3260576848625 > 430.4556724679952.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -208.63557894463653 (T) = (0 -716.1692256534898) / Math.Sqrt((8.85000334944311 / (29)) + (34.44625753852311 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7360626164575006 = (412.5249970043473 - 716.1692256534898) / 412.5249970043473 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 529.398674948285 > 255.53551019181268.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -110.16580984757552 (T) = (0 -546.4483977152572) / Math.Sqrt((2.2782712429882004 / (30)) + (140.78396760351174 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.2300723571965495 = (245.0361738047836 - 546.4483977152572) / 245.0361738047836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 723.7297341551201 > 435.81265805055614.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -233.48888521640708 (T) = (0 -719.6477907361157) / Math.Sqrt((8.245170379692249 / (29)) + (26.779954453078297 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7308699787178561 = (415.77229923948164 - 719.6477907361157) / 415.77229923948164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 661.4551905085416 > 389.76758108286697.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -162.9642429524286 (T) = (0 -660.1694403773738) / Math.Sqrt((8.936195851734038 / (30)) + (54.34220892303736 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.781497078850714 = (370.5700380958608 - 660.1694403773738) / 370.5700380958608 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 568.2348762993004 > 292.9958244226837.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -194.9749168585005 (T) = (0 -568.4605798597681) / Math.Sqrt((4.842019612279389 / (29)) + (38.57401781891829 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.0342004539950835 = (279.45160406553623 - 568.4605798597681) / 279.45160406553623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 293.5461081722602 > 243.71332906128725.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.6733715592574 (T) = (0 -295.8138848336759) / Math.Sqrt((3.690718381134191 / (30)) + (23.923458805380548 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.27115446933928744 = (232.71277564514418 - 295.8138848336759) / 232.71277564514418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 708.0064164240437 > 423.78948812854924.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -177.4206122492596 (T) = (0 -710.8325887278559) / Math.Sqrt((5.155133024061168 / (30)) + (53.04852784325422 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.7534650336151905 = (405.3873759103729 - 710.8325887278559) / 405.3873759103729 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 827.0241188611345 > 590.390490444339.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.21922572318153 (T) = (0 -822.0597154765765) / Math.Sqrt((506.361637495302 / (29)) + (180.51528142876938 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.5063722949252141 = (545.7214781803915 - 822.0597154765765) / 545.7214781803915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 293.4812180626177 > 242.91415785587373.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -98.18897806571333 (T) = (0 -294.59890754057034) / Math.Sqrt((1.9861636562912943 / (29)) + (6.439421859002252 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.27022507754340613 = (231.92654022413072 - 294.59890754057034) / 231.92654022413072 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 540.8077186384277 > 258.86845413030056.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -173.8272984860323 (T) = (0 -544.3901323398799) / Math.Sqrt((3.449101117869154 / (29)) + (53.51318405913557 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.2078454894449484 = (246.57075639687963 - 544.3901323398799) / 246.57075639687963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 570.926432363507 > 296.38361488877007.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -161.06727149057758 (T) = (0 -571.040487882139) / Math.Sqrt((3.2510879861996433 / (29)) + (59.165990593107644 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.0268227681443036 = (281.74169782243274 - 571.040487882139) / 281.74169782243274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 290.31094994192176 > 239.4368203329897.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -97.98439956434888 (T) = (0 -291.9953746475138) / Math.Sqrt((1.9987987418954887 / (29)) + (6.79834472559703 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.2807525021352693 = (227.98735443475567 - 291.9953746475138) / 227.98735443475567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 546.8631505668537 > 261.8503266940347.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -421.611436072665 (T) = (0 -548.8950910634682) / Math.Sqrt((3.623398407893288 / (29)) + (7.1472100250239565 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.1917321834144792 = (250.4389428677137 - 548.8950910634682) / 250.4389428677137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 718.1639968199572 > 432.0193223310138.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -219.92708076750415 (T) = (0 -719.2987170445758) / Math.Sqrt((6.91749884562804 / (29)) + (32.15288993292621 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7386813769274522 = (413.7035839859859 - 719.2987170445758) / 413.7035839859859 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 658.4284406613034 > 390.1910284071524.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -191.32489795801365 (T) = (0 -659.584110898439) / Math.Sqrt((5.859418400880204 / (29)) + (39.0713640055059 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7727932994979358 = (372.0592305291523 - 659.584110898439) / 372.0592305291523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 719.5680041561911 > 423.88847290565053.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -119.80615080123255 (T) = (0 -716.8564401597699) / Math.Sqrt((16.59791645851857 / (29)) + (115.64048946537429 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7582786025638705 = (407.7035568279514 - 716.8564401597699) / 407.7035568279514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 629.0769715046407 > 292.4470045372402.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -147.79625776180765 (T) = (0 -624.0993273832895) / Math.Sqrt((3.310402458682111 / (30)) + (102.21094451814456 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.2465315903422898 = (277.80572063453576 - 624.0993273832895) / 277.80572063453576 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDoubles - Duration of single invocation 48.61 ms 82.48 ms 1.70 0.01 False
WriteDoubles - Duration of single invocation 49.91 ms 85.77 ms 1.72 0.02 False
WriteDoubles - Duration of single invocation 49.85 ms 83.06 ms 1.67 0.01 False
WriteDoubles - Duration of single invocation 48.65 ms 81.91 ms 1.68 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.480332625 > 50.99568991586538.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -107.38603434912449 (T) = (0 -83032878.64705995) / Math.Sqrt((166635613898.69904 / (29)) + (1822916817816.406 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.7018823759911285 = (48788846.87827144 - 83032878.64705995) / 48788846.87827144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.77147835416667 > 52.41590097403847.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -132.32708497505362 (T) = (0 -83852496.9144761) / Math.Sqrt((78797333277.40489 / (29)) + (1188105751877.2852 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.6753406878327873 = (50051012.026065744 - 83852496.9144761) / 50051012.026065744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.05829642857144 > 51.988328446875.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -232.9510957738469 (T) = (0 -83450219.80623434) / Math.Sqrt((73180268755.45305 / (30)) + (351672743413.45337 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.6779353451358657 = (49733870.88373016 - 83450219.80623434) / 49733870.88373016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.90797577083333 > 50.996370666875.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -174.3204119095096 (T) = (0 -82477302.25717178) / Math.Sqrt((94848758532.38411 / (29)) + (653109638651.7031 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.6951394763074995 = (48655171.68937097 - 82477302.25717178) / 48655171.68937097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 96.15 μs 110.08 μs 1.14 0.02 False
CtorInvoke - Duration of single invocation 213.19 μs 228.72 μs 1.07 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.07862848968814 > 100.76859571873423.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -43.916810374018496 (T) = (0 -109246.08506658395) / Math.Sqrt((1414388.8391371141 / (30)) + (745004.2870648794 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.1339587389364769 = (96340.44107199548 - 109246.08506658395) / 96340.44107199548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 228.7228982272256 > 224.5835246002416.
IsChangePoint: Marked as a change because one of 12/28/2022 7:19:51 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -31.30006111956664 (T) = (0 -228861.0935245319) / Math.Sqrt((3720179.5959160975 / (29)) + (1681573.111073549 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.06800854160600993 = (214287.69959122586 - 228861.0935245319) / 214287.69959122586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 790.95 ns 976.77 ns 1.23 0.13 False
Parse - Duration of single invocation 782.00 ns 983.12 ns 1.26 0.18 False
Parse - Duration of single invocation 766.85 ns 1.01 μs 1.32 0.17 False
ToStringHebrewIsrael - Duration of single invocation 822.51 ns 1.22 μs 1.48 0.01 False
Parse - Duration of single invocation 820.04 ns 995.55 ns 1.21 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: fr)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 976.7654412650965 > 824.6998596773605.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -21.28964214255558 (T) = (0 -961.0471436520658) / Math.Sqrt((229.7376389035464 / (29)) + (1088.0942442146347 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.21781991452421492 = (789.1537428401582 - 961.0471436520658) / 789.1537428401582 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: )

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 983.1159415038857 > 829.0813930147856.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -20.137782969306947 (T) = (0 -969.2423780520861) / Math.Sqrt((327.20432418364 / (30)) + (1279.502684519439 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.22517295055281125 = (791.1065761080944 - 969.2423780520861) / 791.1065761080944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: da)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0102950782474505 > 810.4584934543169.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -22.840687226834646 (T) = (0 -965.7700496382053) / Math.Sqrt((308.45365963954845 / (29)) + (1041.1661748217184 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.2365686793074441 = (781.0080150009112 - 965.7700496382053) / 781.0080150009112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2205253477025275 > 865.7273915301715.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -79.61135904425478 (T) = (0 -1238.8180767518566) / Math.Sqrt((7.972071074598398 / (30)) + (506.8414318386006 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.5005192263949848 = (825.5929380712646 - 1238.8180767518566) / 825.5929380712646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ja)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 995.5480152466934 > 810.1803671817755.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -22.840894469148918 (T) = (0 -964.6643606825162) / Math.Sqrt((348.5182688659905 / (30)) + (1009.7566360343654 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.2353995653450331 = (780.8521127438606 - 964.6643606825162) / 780.8521127438606 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Divide - Duration of single invocation 99.99 ns 343.65 ns 3.44 0.03 False
Round - Duration of single invocation 18.32 ns 82.62 ns 4.51 0.03 False
Floor - Duration of single invocation 15.93 ns 81.44 ns 5.11 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Divide

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 343.6478415758592 > 105.10067288726107.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -406.5556148723011 (T) = (0 -345.00332714020607) / Math.Sqrt((1.4948895745582822 / (29)) + (5.95120741380804 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -2.4687843629757467 = (99.45943334576151 - 345.00332714020607) / 99.45943334576151 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Round

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.61735292837771 > 18.745194998790915.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -345.09094749439413 (T) = (0 -82.04002270125378) / Math.Sqrt((0.11423169466703073 / (29)) + (0.5835190717814194 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -3.608337989346854 = (17.80251858498804 - 82.04002270125378) / 17.80251858498804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Floor

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.44002823940764 > 16.762471467970496.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -123.21136049885456 (T) = (0 -80.76766587829191) / Math.Sqrt((6.83824536076945 / (29)) + (0.6748315984737807 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -3.8687563612981584 = (16.588972600953234 - 80.76766587829191) / 16.588972600953234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Reverse - Duration of single invocation 27.07 ns 33.44 ns 1.24 0.06 False
Reverse - Duration of single invocation 258.48 ns 352.46 ns 1.36 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.Reverse(Size: 33)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.44181618296213 > 28.995633912773943.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/14/2022 12:26:48 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -19.890097797129027 (T) = (0 -32.922024077984084) / Math.Sqrt((0.06239860259135018 / (29)) + (1.2812604621039663 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.18958717957224464 = (27.675167186840635 - 32.922024077984084) / 27.675167186840635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Reverse(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 352.4628427159068 > 271.24570292576.
IsChangePoint: Marked as a change because one of 11/18/2022 4:41:14 PM, 11/20/2022 12:28:28 PM, 12/7/2022 5:17:51 AM, 12/8/2022 5:23:10 PM, 12/10/2022 11:25:25 PM, 12/14/2022 12:26:48 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -17.932190332092915 (T) = (0 -347.08491764806956) / Math.Sqrt((0.2203709009602575 / (29)) + (463.65198641779875 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.34275229556113684 = (258.48767400767883 - 347.08491764806956) / 258.48767400767883 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseR - Duration of single invocation 113.43 ns 194.93 ns 1.72 0.02 False
ParseO - Duration of single invocation 113.92 ns 183.27 ns 1.61 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ParseR

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 194.92854412630044 > 120.10495007456247.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -35.727289202409054 (T) = (0 -188.1829275957989) / Math.Sqrt((0.5611785364044003 / (29)) + (81.63083024555063 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.6512774608348404 = (113.96202761749004 - 188.1829275957989) / 113.96202761749004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ParseO

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 183.26657187760728 > 119.6188615486548.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -30.35700745001599 (T) = (0 -186.09950481888558) / Math.Sqrt((1.6251601695758866 / (29)) + (106.26814628782353 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.6332044028638736 = (113.94746701181704 - 186.09950481888558) / 113.94746701181704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where01LinqMethodNestedX - Duration of single invocation 611.99 ms 1.70 secs 2.77 0.10 False
Where01LinqQueryX - Duration of single invocation 519.57 ms 1.68 secs 3.23 0.08 False
Count00ForX - Duration of single invocation 459.13 ms 486.26 ms 1.06 0.02 False
Where01LinqMethodX - Duration of single invocation 556.34 ms 1.60 secs 2.88 0.12 False
Where01ForX - Duration of single invocation 483.34 ms 1.56 secs 3.23 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Where01LinqMethodNestedX

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6954227014285714 > 643.8392807394232.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -165.68725283448413 (T) = (0 -1689481804.9536824) / Math.Sqrt((261999530204961.25 / (30)) + (635070528531159.2 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.7530033047182465 = (613686805.9904457 - 1689481804.9536824) / 613686805.9904457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqQueryX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6769561442 > 548.0887164173078.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -81.36812662545115 (T) = (0 -1625890099.85027) / Math.Sqrt((175389318086455.03 / (29)) + (3352267386815088.5 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -2.086830251208962 = (526718338.0794871 - 1625890099.85027) / 526718338.0794871 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Count00ForX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 486.25905735714286 > 478.68463755.
IsChangePoint: Marked as a change because one of 10/24/2022 9:01:44 AM, 11/4/2022 9:06:14 PM, 11/18/2022 6:07:58 AM, 11/25/2022 11:21:20 AM, 12/20/2022 9:29:18 AM, 1/15/2023 6:48:05 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.236036898477572 (T) = (0 -488696582.46073717) / Math.Sqrt((83420666935665.17 / (40)) + (53713045428328.48 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -0.0526241310378636 = (464265038.2514919 - 488696582.46073717) / 464265038.2514919 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.60120332 > 558.2320973425001.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -65.73979211890611 (T) = (0 -1610061685.8463566) / Math.Sqrt((933906030660229.1 / (29)) + (4496169215396954.5 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -2.0255142395133 = (532161331.39249724 - 1610061685.8463566) / 532161331.39249724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01ForX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5632274413076923 > 502.86032890625.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -60.27821614902665 (T) = (0 -1560364529.7767591) / Math.Sqrt((23503186747419.71 / (29)) + (6089388393934017 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -2.2515773362675184 = (479879261.1735631 - 1560364529.7767591) / 479879261.1735631 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 1.63 μs 1.80 μs 1.11 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.Array(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8012535817268085 > 1.712750197253651.
IsChangePoint: Marked as a change because one of 11/9/2022 4:06:23 PM, 11/21/2022 6:40:27 PM, 12/15/2022 2:06:03 PM, 12/17/2022 10:44:50 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -40.24174646474979 (T) = (0 -1800.6761195115384) / Math.Sqrt((625.3353470755839 / (29)) + (7.046511772427281 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.11690044130663996 = (1612.208262184016 - 1800.6761195115384) / 1612.208262184016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 22.92 μs 24.70 μs 1.08 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.ImmutableList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.70195680446194 > 24.047968804885972.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/8/2022 11:25:18 AM, 12/8/2022 9:08:34 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -30.69356993549545 (T) = (0 -24894.319010337862) / Math.Sqrt((104406.92945655865 / (29)) + (9937.31436616675 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.08597820748280374 = (22923.405680525186 - 24894.319010337862) / 22923.405680525186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 935.04 ns 1.21 μs 1.29 0.06 False
Parse - Duration of single invocation 942.37 ns 1.21 μs 1.28 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.TryParse(value: "12/30/2017 3:45:22 AM -08:00")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2082071049413867 > 982.3357440060887.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -22.82292151115606 (T) = (0 -1179.4367479030932) / Math.Sqrt((371.67376099961524 / (29)) + (1738.8958640123828 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.24635251880360665 = (946.310718764586 - 1179.4367479030932) / 946.310718764586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.Parse(value: "12/30/2017 3:45:22 AM -08:00")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2066760680746635 > 1001.3101402732747.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -13.924955729142031 (T) = (0 -1168.3205950497459) / Math.Sqrt((407.7397616199037 / (30)) + (3993.993975463496 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.21699645192347988 = (960.0032877689979 - 1168.3205950497459) / 960.0032877689979 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 3.40 μs 4.03 μs 1.19 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<String>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.030266953467388 > 3.652109134442374.
IsChangePoint: Marked as a change because one of 11/14/2022 3:36:58 AM, 12/6/2022 2:12:43 AM, 12/13/2022 4:26:35 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -13.725878897677497 (T) = (0 -4044.7204929833274) / Math.Sqrt((37230.91714339289 / (30)) + (870.9075396570827 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.13860965346845566 = (3552.3328654927686 - 4044.7204929833274) / 3552.3328654927686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 260.75 ns 568.11 ns 2.18 0.01 False
ToString - Duration of single invocation 258.76 ns 566.99 ns 2.19 0.02 False
ToString - Duration of single invocation 265.79 ns 569.39 ns 2.14 0.01 False
ToString - Duration of single invocation 259.98 ns 580.80 ns 2.23 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 568.1088069975424 > 273.23605885238703.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -203.28124103726992 (T) = (0 -565.7886076355927) / Math.Sqrt((3.7057208297224244 / (29)) + (40.23034261181531 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.166160608358249 = (261.1942094471049 - 565.7886076355927) / 261.1942094471049 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 566.9901680798765 > 272.81484418246765.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -202.9058766954839 (T) = (0 -568.279923037011) / Math.Sqrt((8.203468787657885 / (29)) + (38.088181208064526 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.1740361060479487 = (261.3939674028935 - 568.279923037011) / 261.3939674028935 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 569.3888335039048 > 274.02648395512426.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -135.77940786649023 (T) = (0 -569.8785269583605) / Math.Sqrt((5.33775660457335 / (29)) + (94.14404685488279 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.174486808619558 = (262.07495244367095 - 569.8785269583605) / 262.07495244367095 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 580.797933263012 > 273.5373796324488.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -150.3042396591316 (T) = (0 -567.8990462363707) / Math.Sqrt((5.130018246659897 / (29)) + (75.5936813363499 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.1716751745781921 = (261.5027573571976 - 567.8990462363707) / 261.5027573571976 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Insert - Duration of single invocation 30.57 ns 34.62 ns 1.13 0.03 False
Format_OneArg - Duration of single invocation 2.60 μs 3.75 μs 1.44 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.61615250484847 > 32.22469504818884.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -12.188873801463547 (T) = (0 -32.91185381905294) / Math.Sqrt((0.29197116381590177 / (29)) + (0.3511622177128073 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.06675445027932529 = (30.852323897439668 - 32.91185381905294) / 30.852323897439668 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7468113857998286 > 2.711551400797853.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -85.76866837536454 (T) = (0 -3656.4526954097046) / Math.Sqrt((1579.6693218133926 / (29)) + (2051.5334194409984 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.4264678197943639 = (2563.2914003884152 - 3656.4526954097046) / 2563.2914003884152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 16.52 μs 19.02 μs 1.15 0.44 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.017824547604583 > 17.917311479031888.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -17.500628898275764 (T) = (0 -18940.549277595906) / Math.Sqrt((344810.5642613765 / (29)) + (14443.37806567276 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.11597494618070898 = (16972.199369186266 - 18940.549277595906) / 16972.199369186266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 3.25 μs 3.61 μs 1.11 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6060831824171995 > 3.5005625977423205.
IsChangePoint: Marked as a change because one of 12/14/2022 12:26:48 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -10.188837341499802 (T) = (0 -3715.998742800695) / Math.Sqrt((3856.6730404138025 / (29)) + (28854.996283241722 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.12540664403627944 = (3301.9164783613123 - 3715.998742800695) / 3301.9164783613123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 298.69 ns 654.03 ns 2.19 0.01 False
ToStringWithFormat - Duration of single invocation 702.13 ns 763.61 ns 1.09 0.03 False
ToStringWithFormat - Duration of single invocation 506.92 ns 780.08 ns 1.54 0.03 False
ToStringWithFormat - Duration of single invocation 258.25 ns 601.56 ns 2.33 0.00 False
ToString - Duration of single invocation 296.41 ns 653.55 ns 2.20 0.01 False
TryParse - Duration of single invocation 165.69 ns 233.08 ns 1.41 0.02 False
ToStringWithFormat - Duration of single invocation 253.39 ns 600.64 ns 2.37 0.01 False
ToStringWithFormat - Duration of single invocation 325.81 ns 662.79 ns 2.03 0.01 False
ToStringWithFormat - Duration of single invocation 325.71 ns 671.43 ns 2.06 0.00 False
Parse - Duration of single invocation 169.27 ns 231.61 ns 1.37 0.01 False
ToStringWithFormat - Duration of single invocation 300.38 ns 662.02 ns 2.20 0.01 False
ToStringWithFormat - Duration of single invocation 2.41 μs 2.61 μs 1.09 0.04 False
ToStringWithFormat - Duration of single invocation 250.27 ns 545.08 ns 2.18 0.01 False
ToStringWithFormat - Duration of single invocation 302.07 ns 659.08 ns 2.18 0.01 False
ToString - Duration of single invocation 300.67 ns 654.15 ns 2.18 0.02 False
ToStringWithFormat - Duration of single invocation 305.25 ns 662.52 ns 2.17 0.00 False
ToStringWithCultureInfo - Duration of single invocation 242.39 ns 552.12 ns 2.28 0.01 False
TryParse - Duration of single invocation 169.21 ns 234.61 ns 1.39 0.02 False
Parse - Duration of single invocation 167.76 ns 244.06 ns 1.45 0.02 False
ToString - Duration of single invocation 245.47 ns 542.08 ns 2.21 0.01 False
ToStringWithFormat - Duration of single invocation 277.83 ns 626.88 ns 2.26 0.01 False
ToStringWithCultureInfo - Duration of single invocation 293.90 ns 654.09 ns 2.23 0.00 False
ToStringWithCultureInfo - Duration of single invocation 290.99 ns 676.68 ns 2.33 0.00 False
ToStringWithFormat - Duration of single invocation 2.42 μs 2.62 μs 1.08 0.04 False
ToStringWithFormat - Duration of single invocation 250.26 ns 545.46 ns 2.18 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 654.0284086933401 > 314.4320071835497.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -265.6010833567599 (T) = (0 -658.7624856202686) / Math.Sqrt((3.1796811666419322 / (29)) + (32.73798745450293 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.2017505513810058 = (299.1994189382955 - 658.7624856202686) / 299.1994189382955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "F50")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 763.6137461525308 > 739.7071644341968.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 12/6/2022 2:12:43 AM, 12/15/2022 2:06:03 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -16.369347310890948 (T) = (0 -761.421139187186) / Math.Sqrt((398.6417234421662 / (29)) + (66.16386983441525 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.097977054730501 = (693.4763672033904 - 761.421139187186) / 693.4763672033904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 780.079824294798 > 534.0631846145369.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -67.7112584870917 (T) = (0 -785.548554912342) / Math.Sqrt((337.88564056592026 / (29)) + (115.9349648344015 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.57030790647002 = (500.2512893654207 - 785.548554912342) / 500.2512893654207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 601.5560872116977 > 270.15979445273786.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -406.85887383878196 (T) = (0 -603.9725402081265) / Math.Sqrt((6.509759064148381 / (30)) + (9.65201929810276 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.34510214224735 = (257.5463683766583 - 603.9725402081265) / 257.5463683766583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 653.5480824191987 > 311.5335530760805.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -214.67399288465037 (T) = (0 -655.6762058204686) / Math.Sqrt((3.3304253427184762 / (29)) + (51.40402479152899 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.221449004070028 = (295.15699195397747 - 655.6762058204686) / 295.15699195397747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 233.08461641483171 > 174.293827616092.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -37.53806218353116 (T) = (0 -235.86691445310703) / Math.Sqrt((1.2336313394920204 / (29)) + (64.96881928718254 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.4206932873484833 = (166.02240367681205 - 235.86691445310703) / 166.02240367681205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 600.6361556343065 > 265.8260103684964.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -125.11498834467208 (T) = (0 -605.1703579014272) / Math.Sqrt((3.2837065262889675 / (30)) + (147.77329366476175 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.3844387511777225 = (253.7999173191265 - 605.1703579014272) / 253.7999173191265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 662.7944243840273 > 342.9895355800179.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -287.34342089320734 (T) = (0 -670.2796502886567) / Math.Sqrt((15.37672782504564 / (29)) + (17.227190432168157 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.0571089661122461 = (325.8357536379932 - 670.2796502886567) / 325.8357536379932 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 671.4280810691715 > 341.60097786904953.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -169.51504862899273 (T) = (0 -669.9137886271908) / Math.Sqrt((9.454414035789485 / (29)) + (73.44542496530109 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.0749331065079422 = (322.860426934263 - 669.9137886271908) / 322.860426934263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 231.61364372016712 > 173.91217993679913.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -87.71542586536157 (T) = (0 -232.263467090006) / Math.Sqrt((4.83133007826297 / (29)) + (7.899595499701093 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.4048877255476597 = (165.3252874705443 - 232.263467090006) / 165.3252874705443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 662.017656400658 > 316.44772016496984.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -221.07728877142569 (T) = (0 -660.5774974758059) / Math.Sqrt((3.6544895259184402 / (30)) + (48.14870261315761 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.199832464906167 = (300.2853662785555 - 660.5774974758059) / 300.2853662785555 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "F50")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6145128186315616 > 2.5333567985220498.
IsChangePoint: Marked as a change because one of 11/11/2022 8:39:50 AM, 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -29.343355331023314 (T) = (0 -2625.0517310905457) / Math.Sqrt((140.06361298302957 / (18)) + (1376.4353805909832 / (30))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (30) - 2, .025) and -0.08918883749814902 = (2410.0979010400542 - 2625.0517310905457) / 2410.0979010400542 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 545.0759629398748 > 263.78826260071423.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -484.30982747285145 (T) = (0 -549.5658999942694) / Math.Sqrt((2.522196353447305 / (29)) + (5.5401397414206 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.1844227946522923 = (251.58403461988556 - 549.5658999942694) / 251.58403461988556 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 659.0753522914816 > 318.0330506442254.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -503.9978964320772 (T) = (0 -660.2208830146675) / Math.Sqrt((2.9731011968090955 / (29)) + (7.625448709139615 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.1827827365451937 = (302.46752091306814 - 660.2208830146675) / 302.46752091306814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 654.1542721430271 > 314.0705638461883.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -130.58999193946028 (T) = (0 -662.2534623647074) / Math.Sqrt((6.103339518549755 / (29)) + (143.24649116756595 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.217028519816621 = (298.7121980819105 - 662.2534623647074) / 298.7121980819105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 662.5209124573404 > 318.88720632521563.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -229.64802350706734 (T) = (0 -665.5647171054738) / Math.Sqrt((4.304403865070362 / (29)) + (44.26149724618399 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.1889115743572685 = (304.06194791167115 - 665.5647171054738) / 304.06194791167115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 552.1190175091601 > 254.9955320211635.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -107.2045458086955 (T) = (0 -546.2691252381959) / Math.Sqrt((4.609782099871686 / (30)) + (147.88392622427017 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.236564780607232 = (244.2447140251769 - 546.2691252381959) / 244.2447140251769 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 234.61319983961343 > 177.67120613058867.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -130.8616959635135 (T) = (0 -234.95011496271817) / Math.Sqrt((2.999071755898825 / (30)) + (2.752750459227983 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.38046283843060175 = (170.1966242204857 - 234.95011496271817) / 170.1966242204857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 244.05685909129028 > 175.82511777996942.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -71.14913666796858 (T) = (0 -234.95030920793602) / Math.Sqrt((2.1519102887298387 / (29)) + (15.781540928011967 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.40459919937357136 = (167.27213664419008 - 234.95030920793602) / 167.27213664419008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 542.0763718029123 > 259.361801862458.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -118.9934172123612 (T) = (0 -549.1406213559413) / Math.Sqrt((2.5862249933046875 / (29)) + (120.75917804667428 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.222758097017966 = (247.05370417620517 - 549.1406213559413) / 247.05370417620517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 626.8805623204427 > 292.8778812361079.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -207.56255936593305 (T) = (0 -626.8628127111009) / Math.Sqrt((3.2842096059821624 / (30)) + (51.06621967210572 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.241038240223065 = (279.71981979598223 - 626.8628127111009) / 279.71981979598223 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 654.0872979037431 > 310.4694275506422.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -257.32229620333663 (T) = (0 -660.9560697864897) / Math.Sqrt((5.249942719405927 / (29)) + (34.985238448807614 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.24038690628474 = (295.01871660308933 - 660.9560697864897) / 295.01871660308933 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 676.6796864212979 > 305.19404277652853.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -196.97150784803912 (T) = (0 -655.7380118631711) / Math.Sqrt((2.1050530581877416 / (29)) + (63.94878967653666 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -1.2572749823289673 = (290.4998358625348 - 655.7380118631711) / 290.4998358625348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "F50")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6183886098813898 > 2.534799302298246.
IsChangePoint: Marked as a change because one of 11/8/2022 11:25:18 AM, 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.78263929704055 (T) = (0 -2620.2566505093628) / Math.Sqrt((87.4442787438424 / (18)) + (1901.4312208273884 / (30))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (30) - 2, .025) and -0.08847454185913332 = (2407.2741710926184 - 2620.2566505093628) / 2407.2741710926184 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 545.4622587845622 > 262.6944040917442.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -169.7664475596739 (T) = (0 -550.5873454629982) / Math.Sqrt((2.3114551387649938 / (30)) + (58.01297161103727 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -1.200391618831693 = (250.22243347542602 - 550.5873454629982) / 250.22243347542602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableStack - Duration of single invocation 6.59 μs 7.17 μs 1.09 0.03 False
Span - Duration of single invocation 1.46 μs 1.69 μs 1.15 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.ImmutableStack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.166748529240447 > 6.98062588226181.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/8/2022 11:25:18 AM, 12/6/2022 2:12:43 AM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -22.690147785200242 (T) = (0 -7205.766264655771) / Math.Sqrt((1224.9760694074084 / (30)) + (10439.213693243079 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.08284099337202241 = (6654.500807377679 - 7205.766264655771) / 6654.500807377679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6867704887690924 > 1.5395836296572023.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 4:06:23 PM, 11/21/2022 6:40:27 PM, 12/14/2022 12:26:48 AM, 12/20/2022 9:29:18 AM, 1/9/2023 10:41:51 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -103.08856809859162 (T) = (0 -1691.2127698124075) / Math.Sqrt((67.90547649484006 / (19)) + (35.2120232556461 / (29))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (29) - 2, .025) and -0.15391106230588397 = (1465.6352859923386 - 1691.2127698124075) / 1465.6352859923386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSingle - Duration of single invocation 128.38 ns 191.39 ns 1.49 0.01 False
TryParseDouble - Duration of single invocation 155.15 ns 219.42 ns 1.41 0.01 False
TryParseSingle - Duration of single invocation 126.32 ns 191.42 ns 1.52 0.01 False
TryParseDouble - Duration of single invocation 153.80 ns 218.47 ns 1.42 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 191.38943702329416 > 134.5246770123366.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -236.35753764170667 (T) = (0 -192.39822913858873) / Math.Sqrt((0.4376334039004304 / (29)) + (1.1204594547723152 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.502231618186657 = (128.07494317742595 - 192.39822913858873) / 128.07494317742595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: -1.7976931348623157e+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.41742874523302 > 162.7816443893377.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -200.45229072702938 (T) = (0 -219.36646712776255) / Math.Sqrt((0.42123597735398993 / (29)) + (1.7055667595366726 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.41863707929334837 = (154.63184371089002 - 219.36646712776255) / 154.63184371089002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 191.42169178958437 > 132.55324956703842.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -102.71121368318308 (T) = (0 -191.20793550837166) / Math.Sqrt((0.5059701633409832 / (29)) + (7.241387567279164 / (19))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (19) - 2, .025) and -0.5131562317678293 = (126.36364407988613 - 191.20793550837166) / 126.36364407988613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 218.47049151135198 > 161.2415521355118.
IsChangePoint: Marked as a change because one of 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -89.5490271190974 (T) = (0 -218.65378415394486) / Math.Sqrt((0.4397371429520371 / (30)) + (9.886900469388795 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.4276817688088297 = (153.15302676756613 - 218.65378415394486) / 153.15302676756613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline db717e30839c532cad5b269ac11c5d2c91dad639
Compare 632f2cd18ac052eb2b4b89cb595221fd4b59a4f4
Diff Diff

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 31.92 ns 34.89 ns 1.09 0.08 False
ToString - Duration of single invocation 34.48 ns 86.02 ns 2.50 0.03 False
TryParse - Duration of single invocation 26.76 ns 29.57 ns 1.11 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.TryParse(value: "255")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.89467392863845 > 33.529507323643166.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/17/2022 10:44:50 PM, 1/11/2023 4:46:06 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -13.939549122308618 (T) = (0 -34.67911313417391) / Math.Sqrt((0.49512199910210236 / (30)) + (0.14843928924512795 / (19))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (19) - 2, .025) and -0.06687211787941894 = (32.50540768007347 - 34.67911313417391) / 32.50540768007347 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Byte.ToString(value: 255)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.0190854622898 > 36.159223586284476.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -208.09088020455494 (T) = (0 -86.63277922093681) / Math.Sqrt((0.05996396035017783 / (39)) + (0.5502022059378066 / (9))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (9) - 2, .025) and -1.5082674299673235 = (34.53889253829103 - 86.63277922093681) / 34.53889253829103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Byte.TryParse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.574930118345986 > 28.120408992520655.
IsChangePoint: Marked as a change because one of 12/15/2022 2:06:03 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/9/2023 6:56:09 PM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -12.481163122343256 (T) = (0 -29.29737289608441) / Math.Sqrt((0.39236983149566795 / (21)) + (0.11831601273959559 / (27))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (27) - 2, .025) and -0.069177634208239 = (27.401782415491766 - 29.29737289608441) / 27.401782415491766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added LLVM=false PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Jan 17, 2023
@lewing
Copy link
Member

lewing commented Jan 29, 2023 •
edited
Loading

looks like dotnet/runtime#79794 and dotnet/runtime#76642 (which should be fixed)

@naricc
Copy link

naricc commented Jan 31, 2023

I am assigning to myself while I try to figure out where this goes.

@naricc naricc self-assigned this Jan 31, 2023
@naricc naricc removed the untriaged label Jan 31, 2023
@naricc naricc mentioned this issue Jan 31, 2023
Closed
@naricc naricc added the LinkedFromMaster Is this issue linked from the master perf issue label Jan 31, 2023
@kotlarmilos
Copy link
Member

Mitigated by dotnet/runtime#81695

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@naricc naricc

Labels
arch-x64 branch-refs/heads/main kind-micro_mono LinkedFromMaster Is this issue linked from the master perf issue mono-aot os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lewing @kotlarmilos @naricc