Several Arm64 regressions (benchmarks turned bimodal)

[performanceautofiler]

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Threading.Tests.Perf_Interlocked

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CompareExchange_object_NoMatch - Duration of single invocation	3.54 ns	7.32 ns	2.07	0.41	False
CompareExchange_object_Match - Duration of single invocation	3.48 ns	7.64 ns	2.20	0.42	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Interlocked*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Interlocked.CompareExchange_object_NoMatch

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.816666186272952 (T) = (0 -6.783821506976851) / Math.Sqrt((0.05449950977022979 / (10)) + (1.3800009198756296 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.8344712154410038 = (3.697971082825648 - 6.783821506976851) / 3.697971082825648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_Interlocked.CompareExchange_object_Match

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -25.852107526032157 (T) = (0 -7.475512500952149) / Math.Sqrt((0.13311071732212948 / (10)) + (0.040929503455113696 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -1.028568926197658 = (3.685116342072843 - 7.475512500952149) / 3.685116342072843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TrimEnd - Duration of single invocation	2.71 ns	4.09 ns	1.51	0.40	False
Trim - Duration of single invocation	3.48 ns	5.40 ns	1.55	0.40	False
TrimStart - Duration of single invocation	2.26 ns	3.46 ns	1.53	0.36	False
ToLowerInvariant - Duration of single invocation	18.49 ns	20.13 ns	1.09	0.29	False
ToUpperInvariant - Duration of single invocation	5.30 ns	7.07 ns	1.34	0.36	False
Remove_IntInt - Duration of single invocation	13.41 ns	15.56 ns	1.16	0.25	False
Remove_Int - Duration of single invocation	2.95 ns	4.58 ns	1.55	0.37	False
ToLowerInvariant - Duration of single invocation	5.15 ns	6.88 ns	1.34	0.33	False
Trim_CharArr - Duration of single invocation	7.41 ns	9.32 ns	1.26	0.18	False
Replace_Char - Duration of single invocation	5.08 ns	6.94 ns	1.37	0.34	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.TrimEnd(s: "Test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/3/2022 2:32:50 PM, 2/4/2022 8:03:09 AM, 3/14/2022 2:55:27 AM, 3/16/2022 7:30:14 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.326147391614285 (T) = (0 -3.8349137599111103) / Math.Sqrt((0.027753478059690695 / (10)) + (0.2959814488284097 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.528452601663384 = (2.509017129963763 - 3.8349137599111103) / 2.509017129963763 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/14/2022 9:07:48 PM, 2/15/2022 10:33:05 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -23.94373519541076 (T) = (0 -5.144860686779068) / Math.Sqrt((0.009302133286992552 / (10)) + (0.0210015629970575 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5000353685407176 = (3.4298262525530667 - 5.144860686779068) / 3.4298262525530667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/8/2022 11:58:21 PM, 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -15.605698726062618 (T) = (0 -3.532241054461829) / Math.Sqrt((0.020353921058750467 / (10)) + (0.01944179112807281 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5152455951888241 = (2.3311343492284857 - 3.532241054461829) / 2.3311343492284857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/20/2022 8:03:17 PM, 2/21/2022 7:08:33 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -4.315187196366555 (T) = (0 -20.8569239607393) / Math.Sqrt((0.5486743995687453 / (10)) + (0.5584055095364985 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.09222111487750406 = (19.09588056542788 - 20.8569239607393) / 19.09588056542788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.046342447446288 (T) = (0 -6.76151079230412) / Math.Sqrt((0.027544699046910093 / (10)) + (0.35619295436522647 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.25472901064263054 = (5.388821598092403 - 6.76151079230412) / 5.388821598092403 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -3.0939117834899 (T) = (0 -15.752755404502121) / Math.Sqrt((0.5087123688840144 / (10)) + (1.1095169822797968 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.1143027236973657 = (14.136872386197652 - 15.752755404502121) / 14.136872386197652 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/27/2022 5:13:24 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.392795965671501 (T) = (0 -4.264441251973207) / Math.Sqrt((0.019686255364293297 / (10)) + (0.2763457922039334 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.4337889544319503 = (2.974246132104377 - 4.264441251973207) / 2.974246132104377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/22/2022 8:24:54 PM, 1/24/2022 2:43:25 PM, 3/7/2022 3:58:30 PM, 3/8/2022 5:51:58 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -45.4824367722818 (T) = (0 -6.932147705710385) / Math.Sqrt((0.009931324927593617 / (10)) + (0.0023624363254665334 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.33543598055299656 = (5.190924766636751 - 6.932147705710385) / 5.190924766636751 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2022 8:22:24 PM, 2/3/2022 2:32:50 PM, 5/24/2022 10:45:34 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -6.213126865364811 (T) = (0 -8.913989332631168) / Math.Sqrt((0.03617540558944572 / (11)) + (0.16551080531201065 / (4))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (4) - 2, .025) and -0.1727573934479034 = (7.6008809515359905 - 8.913989332631168) / 7.6008809515359905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -9.046790679205726 (T) = (0 -6.917504257976644) / Math.Sqrt((0.01863522138750699 / (10)) + (0.18627555603558055 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.34891704543587265 = (5.12819100431888 - 6.917504257976644) / 5.12819100431888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayAnd - Duration of single invocation	3.18 ns	4.97 ns	1.56	0.45	False
BitArrayNot - Duration of single invocation	1.59 ns	3.05 ns	1.92	0.47	False
BitArrayXor - Duration of single invocation	3.18 ns	4.94 ns	1.55	0.42	False
BitArrayOr - Duration of single invocation	3.26 ns	5.35 ns	1.64	0.45	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -4.420649253718811 (T) = (0 -4.5543078059637185) / Math.Sqrt((0.023742319640434374 / (10)) + (0.5950948452821324 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5110078667615688 = (3.014086098522193 - 4.5543078059637185) / 3.014086098522193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2022 1:56:18 PM, 2/19/2022 11:51:48 PM, 3/12/2022 3:27:04 PM, 3/13/2022 11:50:20 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -3.2768161461655296 (T) = (0 -2.6073506115431586) / Math.Sqrt((0.024241548191758795 / (10)) + (0.3675053446801315 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5297410000528185 = (1.7044392556995809 - 2.6073506115431586) / 1.7044392556995809 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 10:45:34 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -6.554115773683309 (T) = (0 -4.743384000189914) / Math.Sqrt((0.08937230356401307 / (11)) + (0.1966556163855758 / (4))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (4) - 2, .025) and -0.49414243786408185 = (3.17465315219258 - 4.743384000189914) / 3.17465315219258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -21.22569288530968 (T) = (0 -5.096478364263296) / Math.Sqrt((0.02145283227719352 / (10)) + (0.03625636512313969 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.6770534479653864 = (3.038948085075274 - 5.096478364263296) / 3.038948085075274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_Byte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	1.62 ns	2.71 ns	1.68	0.41	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/26/2022 3:32:05 AM, 2/27/2022 3:52:50 PM, 3/11/2022 5:51:22 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -13.084683436605099 (T) = (0 -2.9010167376618305) / Math.Sqrt((0.023353574352051314 / (10)) + (0.0297598977897629 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.6966448429165885 = (1.7098550411262778 - 2.9010167376618305) / 1.7098550411262778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	1.61 ns	2.71 ns	1.68	0.45	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/11/2022 5:51:22 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -12.865793830423286 (T) = (0 -2.9065699779197303) / Math.Sqrt((0.024036310872644937 / (10)) + (0.031299750448966976 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.7007003376775989 = (1.709042982780149 - 2.9065699779197303) / 1.709042982780149 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetDocumentElement - Duration of single invocation	5.12 ns	6.77 ns	1.32	0.50	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -7.253457349786897 (T) = (0 -6.619961155732513) / Math.Sqrt((0.027087059643368994 / (10)) + (0.23612120939975262 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.32422352429225165 = (4.999126683896238 - 6.619961155732513) / 4.999126683896238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	1.67 ns	2.78 ns	1.67	0.42	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/15/2022 11:00:34 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -4.546412656016563 (T) = (0 -2.7327756297790544) / Math.Sqrt((0.023739704779073872 / (10)) + (0.23449225106943755 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5855131938529883 = (1.7235905953819786 - 2.7327756297790544) / 1.7235905953819786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

Seems these benchmarks were stable since March and again came back to the bimodal behavior.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Threading.Tests.Perf_Interlocked

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CompareExchange_object_NoMatch - Duration of single invocation	3.54 ns	7.32 ns	2.07	0.41	False
CompareExchange_object_Match - Duration of single invocation	3.48 ns	7.64 ns	2.20	0.42	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tests.Perf_Interlocked*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Interlocked.CompareExchange_object_NoMatch

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.816666186272952 (T) = (0 -6.783821506976851) / Math.Sqrt((0.05449950977022979 / (10)) + (1.3800009198756296 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.8344712154410038 = (3.697971082825648 - 6.783821506976851) / 3.697971082825648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Threading.Tests.Perf_Interlocked.CompareExchange_object_Match

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -25.852107526032157 (T) = (0 -7.475512500952149) / Math.Sqrt((0.13311071732212948 / (10)) + (0.040929503455113696 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -1.028568926197658 = (3.685116342072843 - 7.475512500952149) / 3.685116342072843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TrimEnd - Duration of single invocation	2.71 ns	4.09 ns	1.51	0.40	False
Trim - Duration of single invocation	3.48 ns	5.40 ns	1.55	0.40	False
TrimStart - Duration of single invocation	2.26 ns	3.46 ns	1.53	0.36	False
ToLowerInvariant - Duration of single invocation	18.49 ns	20.13 ns	1.09	0.29	False
ToUpperInvariant - Duration of single invocation	5.30 ns	7.07 ns	1.34	0.36	False
Remove_IntInt - Duration of single invocation	13.41 ns	15.56 ns	1.16	0.25	False
Remove_Int - Duration of single invocation	2.95 ns	4.58 ns	1.55	0.37	False
ToLowerInvariant - Duration of single invocation	5.15 ns	6.88 ns	1.34	0.33	False
Trim_CharArr - Duration of single invocation	7.41 ns	9.32 ns	1.26	0.18	False
Replace_Char - Duration of single invocation	5.08 ns	6.94 ns	1.37	0.34	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.TrimEnd(s: "Test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/3/2022 2:32:50 PM, 2/4/2022 8:03:09 AM, 3/14/2022 2:55:27 AM, 3/16/2022 7:30:14 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.326147391614285 (T) = (0 -3.8349137599111103) / Math.Sqrt((0.027753478059690695 / (10)) + (0.2959814488284097 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.528452601663384 = (2.509017129963763 - 3.8349137599111103) / 2.509017129963763 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/14/2022 9:07:48 PM, 2/15/2022 10:33:05 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -23.94373519541076 (T) = (0 -5.144860686779068) / Math.Sqrt((0.009302133286992552 / (10)) + (0.0210015629970575 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5000353685407176 = (3.4298262525530667 - 5.144860686779068) / 3.4298262525530667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.TrimStart(s: "Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/8/2022 11:58:21 PM, 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -15.605698726062618 (T) = (0 -3.532241054461829) / Math.Sqrt((0.020353921058750467 / (10)) + (0.01944179112807281 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5152455951888241 = (2.3311343492284857 - 3.532241054461829) / 2.3311343492284857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "TeSt")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/20/2022 8:03:17 PM, 2/21/2022 7:08:33 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -4.315187196366555 (T) = (0 -20.8569239607393) / Math.Sqrt((0.5486743995687453 / (10)) + (0.5584055095364985 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.09222111487750406 = (19.09588056542788 - 20.8569239607393) / 19.09588056542788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "TEST")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/15/2022 7:21:49 PM, 3/16/2022 12:48:00 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.046342447446288 (T) = (0 -6.76151079230412) / Math.Sqrt((0.027544699046910093 / (10)) + (0.35619295436522647 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.25472901064263054 = (5.388821598092403 - 6.76151079230412) / 5.388821598092403 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -3.0939117834899 (T) = (0 -15.752755404502121) / Math.Sqrt((0.5087123688840144 / (10)) + (1.1095169822797968 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.1143027236973657 = (14.136872386197652 - 15.752755404502121) / 14.136872386197652 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Remove_Int(s: "dzsdzsDDZSDZSDZSddsz", i: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/27/2022 5:13:24 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -5.392795965671501 (T) = (0 -4.264441251973207) / Math.Sqrt((0.019686255364293297 / (10)) + (0.2763457922039334 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.4337889544319503 = (2.974246132104377 - 4.264441251973207) / 2.974246132104377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/22/2022 8:24:54 PM, 1/24/2022 2:43:25 PM, 3/7/2022 3:58:30 PM, 3/8/2022 5:51:58 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -45.4824367722818 (T) = (0 -6.932147705710385) / Math.Sqrt((0.009931324927593617 / (10)) + (0.0023624363254665334 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.33543598055299656 = (5.190924766636751 - 6.932147705710385) / 5.190924766636751 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2022 8:22:24 PM, 2/3/2022 2:32:50 PM, 5/24/2022 10:45:34 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -6.213126865364811 (T) = (0 -8.913989332631168) / Math.Sqrt((0.03617540558944572 / (11)) + (0.16551080531201065 / (4))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (4) - 2, .025) and -0.1727573934479034 = (7.6008809515359905 - 8.913989332631168) / 7.6008809515359905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/13/2022 11:50:20 AM, 3/14/2022 10:40:48 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -9.046790679205726 (T) = (0 -6.917504257976644) / Math.Sqrt((0.01863522138750699 / (10)) + (0.18627555603558055 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.34891704543587265 = (5.12819100431888 - 6.917504257976644) / 5.12819100431888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayAnd - Duration of single invocation	3.18 ns	4.97 ns	1.56	0.45	False
BitArrayNot - Duration of single invocation	1.59 ns	3.05 ns	1.92	0.47	False
BitArrayXor - Duration of single invocation	3.18 ns	4.94 ns	1.55	0.42	False
BitArrayOr - Duration of single invocation	3.26 ns	5.35 ns	1.64	0.45	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -4.420649253718811 (T) = (0 -4.5543078059637185) / Math.Sqrt((0.023742319640434374 / (10)) + (0.5950948452821324 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5110078667615688 = (3.014086098522193 - 4.5543078059637185) / 3.014086098522193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/2/2022 1:56:18 PM, 2/19/2022 11:51:48 PM, 3/12/2022 3:27:04 PM, 3/13/2022 11:50:20 AM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -3.2768161461655296 (T) = (0 -2.6073506115431586) / Math.Sqrt((0.024241548191758795 / (10)) + (0.3675053446801315 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5297410000528185 = (1.7044392556995809 - 2.6073506115431586) / 1.7044392556995809 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 10:45:34 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -6.554115773683309 (T) = (0 -4.743384000189914) / Math.Sqrt((0.08937230356401307 / (11)) + (0.1966556163855758 / (4))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (4) - 2, .025) and -0.49414243786408185 = (3.17465315219258 - 4.743384000189914) / 3.17465315219258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -21.22569288530968 (T) = (0 -5.096478364263296) / Math.Sqrt((0.02145283227719352 / (10)) + (0.03625636512313969 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.6770534479653864 = (3.038948085075274 - 5.096478364263296) / 3.038948085075274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_Byte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	1.62 ns	2.71 ns	1.68	0.41	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/26/2022 3:32:05 AM, 2/27/2022 3:52:50 PM, 3/11/2022 5:51:22 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -13.084683436605099 (T) = (0 -2.9010167376618305) / Math.Sqrt((0.023353574352051314 / (10)) + (0.0297598977897629 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.6966448429165885 = (1.7098550411262778 - 2.9010167376618305) / 1.7098550411262778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	1.61 ns	2.71 ns	1.68	0.45	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/11/2022 5:51:22 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -12.865793830423286 (T) = (0 -2.9065699779197303) / Math.Sqrt((0.024036310872644937 / (10)) + (0.031299750448966976 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.7007003376775989 = (1.709042982780149 - 2.9065699779197303) / 1.709042982780149 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetDocumentElement - Duration of single invocation	5.12 ns	6.77 ns	1.32	0.50	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -7.253457349786897 (T) = (0 -6.619961155732513) / Math.Sqrt((0.027087059643368994 / (10)) + (0.23612120939975262 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.32422352429225165 = (4.999126683896238 - 6.619961155732513) / 4.999126683896238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	e86511ec00d9fad4667c71e73017931b74a049d6
Compare	0dba0ee1b234100ade81d2b4f799cc118150785c
Diff	Diff

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	1.67 ns	2.78 ns	1.67	0.42	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/15/2022 11:00:34 PM, 5/24/2022 3:42:52 PM, 5/26/2022 4:44:18 AM falls between 5/17/2022 2:28:28 PM and 5/26/2022 4:44:18 AM.
IsRegressionStdDev: Marked as regression because -4.546412656016563 (T) = (0 -2.7327756297790544) / Math.Sqrt((0.023739704779073872 / (10)) + (0.23449225106943755 / (5))) is less than -2.160368656462815 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (5) - 2, .025) and -0.5855131938529883 = (1.7235905953819786 - 2.7327756297790544) / 1.7235905953819786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	kunalspathak
Labels:	area-CodeGen-coreclr, refs/heads/main, RunKind=micro, Windows 10.0.19041, Regression, CoreClr, arm64
Milestone:	-

[kunalspathak]

Looking at the e86511e...fd4bff9, the only suspect I can think of is #69672

@janvorli ?

[janvorli]

I've investigated these issues. The codegen with and without W^X enabled is identical in both the benchmarks and the method that calls the benchmarks in a loop. But I've noticed that at least some of the tests were returning managed references and the generated calling loop was calling a GC write barrier for the returned value. So I've tried to temporarily disable the part of the W^X changes to use a copy of write barriers in a block of allocated memory instead of modifying protection of the executable image of coreclr.dll. Interestingly, that completely fixed the regressions. So I have thought that maybe we end up allocating the memory for the barrier code too far from the jitted code and the write barrier call would need to use a less effective way of calling. However, I have found it was not the case, so the only thing I can think of is that the slightly different location of the barriers or the fact that they are now in shared memory is causing the differences.
It is also important to note that the regression is significant only on benchmarks that take couple of nanoseconds to run. The same tests that have variant with e.g. working on longer data sets are ok. So I think we can safely close this issue as won't fix.