[Perf] Windows/x64: 4 Regressions on 5/1/2023 3:42:23 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	79647f40acc3bc3dd0e6fa14e4a158cad6d63c05
Compare	6f19e373de35dbf2c68f8744cc48c1bfeb13644f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfString - Duration of single invocation	13.52 ns	17.98 ns	1.33	0.17	False	157.85552183670094	219.0443083596165	1.3876252525788386)
IndexOfString - Duration of single invocation	21.84 ns	26.56 ns	1.22	0.02	False	274.2468247472681	328.89399839208056	1.1992627396695388)	Trace	Trace
IndexOfString - Duration of single invocation	15.57 ns	23.22 ns	1.49	0.32	False	178.10697362999588	275.84447505433957	1.5487572969904377)
IndexOfString - Duration of single invocation	15.73 ns	20.39 ns	1.30	0.12	False	196.22765866160847	243.14119749776583	1.239077096246962)	Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.975917554957828 > 14.21358965431307.
IsChangePoint: Marked as a change because one of 2/13/2023 3:18:56 PM, 2/14/2023 2:39:38 AM, 5/1/2023 10:17:01 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -144.72784811067038 (T) = (0 -18.029036976403475) / Math.Sqrt((0.006118605047046131 / (47)) + (0.002600087442929838 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.33950389188779057 = (13.459488311747105 - 18.029036976403475) / 13.459488311747105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.563893155055098 > 22.88504729583008.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/1/2023 10:17:01 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -23.881353368669103 (T) = (0 -26.67981608320946) / Math.Sqrt((0.5184512385442349 / (47)) + (0.05863964761829456 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.18556797531886052 = (22.5038265528671 - 26.67981608320946) / 22.5038265528671 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.22372896339271 > 16.355643124760174.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -7.485837754989703 (T) = (0 -24.43695927693157) / Math.Sqrt((0.06257885468195488 / (47)) + (4.479583773291511 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.5987059669183535 = (15.285461981502428 - 24.43695927693157) / 15.285461981502428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.38779001249898 > 16.5478774127217.
IsChangePoint: Marked as a change because one of 4/3/2023 12:37:39 PM, 4/3/2023 9:14:01 PM, 5/1/2023 10:17:01 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -43.36568358897196 (T) = (0 -20.409244354437536) / Math.Sqrt((0.010352626185912775 / (47)) + (0.032493069046758215 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.28761524464379484 = (15.850421497676146 - 20.409244354437536) / 15.850421497676146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository