Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/arm64: 247 Regressions on 7/29/2023 7:04:01 PM #20354

Closed
performanceautofiler bot opened this issue Aug 3, 2023 · 16 comments
Closed

[Perf] Windows/arm64: 247 Regressions on 7/29/2023 7:04:01 PM #20354

performanceautofiler bot opened this issue Aug 3, 2023 · 16 comments
Labels
arch-arm64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.45 ns 13.20 ns 1.26 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.197330507873625 > 10.96757756684213.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.751901600132886 (T) = (0 -12.947852362523099) / Math.Sqrt((0.004496182403397161 / (19)) + (0.1265811912449337 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.2438162228304058 = (10.409779294451715 - 12.947852362523099) / 10.409779294451715 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.28 ns 7.34 ns 1.39 0.10 False
7.44 ns 9.38 ns 1.26 0.49 False
0.71 ns 2.36 ns 3.32 0.65 False
5.61 ns 7.74 ns 1.38 0.28 False
8.76 ns 10.46 ns 1.19 0.14 False
1.09 ns 3.04 ns 2.78 0.60 False
5.11 ns 7.92 ns 1.55 0.42 False
0.81 ns 2.09 ns 2.57 0.56 False
4.04 ns 6.35 ns 1.57 0.38 False
6.75 ns 9.12 ns 1.35 0.14 False
1.98 ns 4.43 ns 2.24 0.53 False
0.79 ns 3.12 ns 3.94 0.56 False
4.20 ns 6.94 ns 1.65 0.55 False
0.81 ns 2.66 ns 3.30 0.66 False
10.77 ns 13.01 ns 1.21 0.12 False
1.99 ns 3.47 ns 1.74 0.52 False
1.21 ns 3.88 ns 3.20 0.70 False
5.30 ns 8.33 ns 1.57 0.10 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Int32>.LastIndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.344581539896516 > 5.636191595837765.
IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.259105491881886 (T) = (0 -7.931729030314941) / Math.Sqrt((0.04173937173157687 / (20)) + (0.1121959538093451 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.44892867959141985 = (5.474202520824966 - 7.931729030314941) / 5.474202520824966 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceEqual(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.382846425361178 > 7.889158324713601.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 9:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.091399921857693 (T) = (0 -10.263239161063579) / Math.Sqrt((0.3030182387703014 / (20)) + (0.16692796742865113 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3720008445052796 = (7.4804904108964525 - 10.263239161063579) / 7.4804904108964525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.356473415761654 > 0.7122043598466549.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -42.14292123021325 (T) = (0 -2.355081214633396) / Math.Sqrt((0.029537538365017817 / (20)) + (0.000870902248579985 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.5219635961337614 = (0.6686841446114572 - 2.355081214633396) / 0.6686841446114572 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.BinarySearch(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.7395513721185845 > 5.860227881953505.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.410083479841392 (T) = (0 -7.869329319720262) / Math.Sqrt((0.24606868370051793 / (20)) + (0.06726530347995366 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.371869019247225 = (5.736210388392864 - 7.869329319720262) / 5.736210388392864 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.460243664390934 > 8.555224693108066.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.397822420130943 (T) = (0 -10.42569128382875) / Math.Sqrt((0.2051159726771844 / (19)) + (0.0009391859735817021 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.2731885521042644 = (8.188646737828165 - 10.42569128382875) / 8.188646737828165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.039012957475028 > 1.0839465080756103.
IsChangePoint: Marked as a change because one of 7/28/2023 9:42:28 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.958340397469567 (T) = (0 -3.0271730678395183) / Math.Sqrt((1.3216954914114623 / (18)) + (0.000827765221898203 / (9))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (9) - 2, .025) and -1.14449767888541 = (1.411600067300084 - 3.0271730678395183) / 1.411600067300084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.EndsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.918359823572989 > 5.115903779174967.
IsChangePoint: Marked as a change because one of 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.406455680633645 (T) = (0 -7.822066425374482) / Math.Sqrt((0.11261705113485392 / (20)) + (0.04928901742046534 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.576379144699183 = (4.962046378041337 - 7.822066425374482) / 4.962046378041337 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfAnyValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0938650947196877 > 1.067432583688914.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/10/2023 2:36:50 PM, 5/11/2023 10:01:37 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.411214708727906 (T) = (0 -2.078135552900125) / Math.Sqrt((0.020242236843419026 / (20)) + (0.0345644852033357 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.1501252757256952 = (0.9665183588889847 - 2.078135552900125) / 0.9665183588889847 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.353145369307167 > 4.282091511083304.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.11387997094202 (T) = (0 -6.14635836027414) / Math.Sqrt((0.6399155553816286 / (20)) + (0.1078389116054198 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5582572406572481 = (3.944379785254008 - 6.14635836027414) / 3.944379785254008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.120370947297962 > 7.271383774244364.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.930576526481023 (T) = (0 -9.124497641107071) / Math.Sqrt((0.2986789732883997 / (19)) + (0.00047048602022555045 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.2810143567639662 = (7.1228691489117395 - 9.124497641107071) / 7.1228691489117395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.EndsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.431707665512283 > 2.0538073493766906.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -60.853196684205976 (T) = (0 -4.443975287472788) / Math.Sqrt((0.03374805365484618 / (20)) + (0.00017458606511115194 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.3075501943881627 = (1.9258412225572799 - 4.443975287472788) / 1.9258412225572799 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.119340236207554 > 0.8477538773524517.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -39.40798536451097 (T) = (0 -3.10829933911099) / Math.Sqrt((0.059835548480784484 / (20)) + (0.0008643374816958634 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.420359897400487 = (0.9087638237933187 - 3.10829933911099) / 0.9087638237933187 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.StartsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.942514457158541 > 4.327803847465658.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -28.844349952518073 (T) = (0 -6.963648779950554) / Math.Sqrt((0.1352108763389609 / (20)) + (0.02571324643406148 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7334128356629123 = (4.017305420083285 - 6.963648779950554) / 4.017305420083285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyThreeValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6595064500567953 > 0.8716542368852565.
IsChangePoint: Marked as a change because one of 3/23/2023 12:23:24 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.361318202633962 (T) = (0 -2.642570340192911) / Math.Sqrt((0.4810334456141768 / (19)) + (0.0009224651121213166 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.6702200066218476 = (0.9896451729219432 - 2.642570340192911) / 0.9896451729219432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.BinarySearch(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.010806926045902 > 11.281356410018521.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/2/2023 2:18:12 PM, 6/16/2023 8:17:59 AM, 7/12/2023 7:14:29 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -85.18687124103596 (T) = (0 -12.98424640532337) / Math.Sqrt((0.0067652389466756335 / (20)) + (0.0026196259828249776 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.21230397419416402 = (10.710388385845379 - 12.98424640532337) / 10.710388385845379 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareToDifferent(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.467766471519646 > 2.309211631984375.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/23/2023 1:41:12 PM, 6/16/2023 8:17:59 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.732807927339739 (T) = (0 -3.307863784593105) / Math.Sqrt((0.2579590788745399 / (20)) + (0.08896007356565155 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5976494296342612 = (2.070456586555632 - 3.307863784593105) / 2.070456586555632 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.StartsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.879425219174133 > 1.2501973989733461.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.1783611017902 (T) = (0 -3.9865403365137793) / Math.Sqrt((0.05704031709876029 / (20)) + (0.04220165693327907 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.2232013848175303 = (1.2368263290317068 - 3.9865403365137793) / 1.2368263290317068 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareTo(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.326230249216097 > 5.606399376836017.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.060395634538665 (T) = (0 -8.304209225317914) / Math.Sqrt((0.1343092051033879 / (20)) + (0.03524728495632238 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5155781943064716 = (5.479235090946871 - 8.304209225317914) / 5.479235090946871 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.93 ns 6.94 ns 1.76 0.43 False
3.78 ns 6.86 ns 1.81 0.39 True
9.50 ns 12.42 ns 1.31 0.08 False
1.74 ns 3.39 ns 1.95 0.60 False
2.64 ns 6.31 ns 2.39 0.51 False
27.06 ns 30.61 ns 1.13 0.01 False
13.49 ns 15.79 ns 1.17 0.07 False
3.93 ns 6.52 ns 1.66 0.40 False
9.84 ns 12.54 ns 1.27 0.04 False
7.16 ns 10.58 ns 1.48 0.29 False
5.45 ns 8.12 ns 1.49 0.34 False
2.97 ns 6.58 ns 2.22 0.49 True
2.53 ns 3.76 ns 1.48 0.56 False
28.06 ns 31.11 ns 1.11 0.03 False
2.13 ns 3.16 ns 1.48 0.70 False
8.51 ns 11.29 ns 1.33 0.16 False
3.53 ns 6.20 ns 1.76 0.43 False
2.61 ns 3.69 ns 1.41 0.55 False
2.10 ns 3.81 ns 1.82 0.63 False
4.17 ns 6.88 ns 1.65 0.39 True
29.50 ns 31.93 ns 1.08 0.02 False
3.87 ns 5.66 ns 1.46 0.45 False
2.03 ns 4.31 ns 2.12 0.62 False
3.69 ns 7.08 ns 1.92 0.37 True
1.85 ns 3.53 ns 1.91 0.61 False
1.60 ns 3.34 ns 2.09 0.65 False
4.27 ns 7.29 ns 1.71 0.45 False
2.19 ns 3.34 ns 1.53 0.57 False
1.25 ns 4.29 ns 3.42 0.63 False
5.07 ns 7.73 ns 1.52 0.42 False
3.17 ns 5.17 ns 1.63 0.54 False
6.17 ns 9.40 ns 1.52 0.31 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.937285219590968 > 3.7622510765185986.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -54.01184241998625 (T) = (0 -6.900125870527781) / Math.Sqrt((0.05818196759558457 / (19)) + (0.0021618858103989247 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.8331125759453151 = (3.7641582743326416 - 6.900125870527781) / 3.7641582743326416 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 65535)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.8567705801134124 > 4.4748982397311465.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -59.4398351197848 (T) = (0 -6.851456495310683) / Math.Sqrt((0.04429210489325353 / (20)) + (4.5193969667161554E-05 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6916476239028514 = (4.050167658145897 - 6.851456495310683) / 4.050167658145897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 2147483647)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.424112697256838 > 10.005422326537325.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.79610933517898 (T) = (0 -12.080547944958218) / Math.Sqrt((0.01942121078889295 / (20)) + (0.17903288679571336 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.27072180458507145 = (9.50683926361276 - 12.080547944958218) / 9.50683926361276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: True )

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3925104857261235 > 2.249671511133489.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.186695658172635 (T) = (0 -3.4175631186580517) / Math.Sqrt((0.04625823195813903 / (19)) + (0.06282193868221465 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6152162907063456 = (2.115854785716362 - 3.4175631186580517) / 2.115854785716362 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 3039)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.311910145103773 > 3.1330695043660617.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -79.15336939710295 (T) = (0 -6.304992459150233) / Math.Sqrt((0.032767760709676574 / (20)) + (0.0011499345681305912 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.1414124188234556 = (2.944314884759261 - 6.304992459150233) / 2.944314884759261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.609736853761717 > 28.8413564190469.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/13/2023 3:47:43 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.588110125691225 (T) = (0 -30.060046498975876) / Math.Sqrt((0.045193726364327134 / (19)) + (0.355486153814707 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.09755544975755287 = (27.388180256055456 - 30.060046498975876) / 27.388180256055456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.791930652179916 > 14.193411333364162.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -41.16859320878564 (T) = (0 -15.782507243404739) / Math.Sqrt((0.03946618946409302 / (19)) + (0.008031186801295416 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.17387716271915032 = (13.444768962747679 - 15.782507243404739) / 13.444768962747679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.521512663772584 > 3.9032522772630434.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -62.232570763963245 (T) = (0 -6.563088616879326) / Math.Sqrt((0.039053244033545216 / (20)) + (0.0021213653352902036 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.8193442530208755 = (3.607392392056557 - 6.563088616879326) / 3.607392392056557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: -2147483648)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.536588515930685 > 10.318985420635526.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -30.486406339205075 (T) = (0 -12.522275854866072) / Math.Sqrt((0.0404429970664157 / (19)) + (0.040788688903447196 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.27738213189858985 = (9.803077358107435 - 12.522275854866072) / 9.803077358107435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.579281851460346 > 7.2710849995268605.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -41.073538693042416 (T) = (0 -10.365839957063669) / Math.Sqrt((0.043288378907829515 / (19)) + (0.032416414965933886 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.491116826795177 = (6.951728912712178 - 10.365839957063669) / 6.951728912712178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: -32768)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.117423970820848 > 5.61029029706459.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -73.85260024374381 (T) = (0 -8.110524275492349) / Math.Sqrt((0.02395961025207004 / (20)) + (0.0012263298800556717 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5092663541914652 = (5.373819043251162 - 8.110524275492349) / 5.373819043251162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 3039)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.579611312465892 > 3.264692256043015.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -73.3095013816752 (T) = (0 -6.553169307435126) / Math.Sqrt((0.035574128181205526 / (19)) + (0.0009632802552471045 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.0061520738507932 = (3.2665366663138196 - 6.553169307435126) / 3.2665366663138196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: true)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.755865760172109 > 2.6465325979730387.
IsChangePoint: Marked as a change because one of 5/20/2023 9:58:12 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.771084354486781 (T) = (0 -3.6381830474688455) / Math.Sqrt((0.019181534079448816 / (20)) + (0.059795887531579224 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.46064936240980575 = (2.490798367560648 - 3.6381830474688455) / 2.490798367560648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.108664033009173 > 29.453150482210948.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.446259481215401 (T) = (0 -30.693127947444648) / Math.Sqrt((0.18712640998395438 / (20)) + (0.6277379781205416 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.08265671484064661 = (28.34982458124992 - 30.693127947444648) / 28.34982458124992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.164722075145689 > 2.134622722512525.
IsChangePoint: Marked as a change because one of 5/25/2023 8:00:22 PM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.133031170129408 (T) = (0 -3.1546362485019697) / Math.Sqrt((0.047769961461392015 / (19)) + (0.03947094024860311 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6023068719779018 = (1.9688090363164072 - 3.1546362485019697) / 1.9688090363164072 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 4294967295)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.294405806743502 > 8.790794749958009.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -66.43281993629003 (T) = (0 -11.320474079643) / Math.Sqrt((0.03211959236240203 / (20)) + (0.0027635680707789084 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3559187546359526 = (8.348932442255663 - 11.320474079643) / 8.348932442255663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: 127)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.19742643478847 > 3.695720838920376.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -59.945295100673455 (T) = (0 -6.16288796500984) / Math.Sqrt((0.030496893834144233 / (20)) + (0.003945275896035056 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.8002187626132464 = (3.4234105837579563 - 6.16288796500984) / 3.4234105837579563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: TRUE)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.687795537837375 > 2.587529340489806.
IsChangePoint: Marked as a change because one of 6/18/2023 4:50:43 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.679556242607059 (T) = (0 -3.438387065997079) / Math.Sqrt((0.055419720415502555 / (20)) + (0.09651009564421313 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.4033123443942621 = (2.4501937004489576 - 3.438387065997079) / 2.4501937004489576 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.808836394998783 > 2.0378320897925013.
IsChangePoint: Marked as a change because one of 5/2/2023 7:50:39 PM, 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.06279743826284 (T) = (0 -3.8510989002008067) / Math.Sqrt((0.20333747124509732 / (19)) + (0.0008720148820712788 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.9532342059498136 = (1.9716523950224927 - 3.8510989002008067) / 1.9716523950224927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.882024562558804 > 4.192510872683027.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -54.23621485022275 (T) = (0 -6.831155761978278) / Math.Sqrt((0.04622873547105635 / (20)) + (0.002152590471980559 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6843840598314298 = (4.05558086477138 - 6.831155761978278) / 4.05558086477138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.931529353792993 > 30.965075494311794.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.592642571252 (T) = (0 -32.197319509328736) / Math.Sqrt((0.05968937528808443 / (19)) + (0.7235288568986458 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.0954028700326387 = (29.39313050035133 - 32.197319509328736) / 29.39313050035133 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: -128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.662566941672113 > 3.9590496703384197.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.24451593112763 (T) = (0 -6.458034033659453) / Math.Sqrt((0.038222201076565133 / (20)) + (0.12373510819411392 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7154374035577102 = (3.7646573522682276 - 6.458034033659453) / 3.7646573522682276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.312750898817867 > 1.3072632098761003.
IsChangePoint: Marked as a change because one of 5/11/2023 7:02:02 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -31.47623101860143 (T) = (0 -4.345699798982653) / Math.Sqrt((0.1907642775551831 / (20)) + (0.004376226920245914 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.7064596389164106 = (1.172466510455008 - 4.345699798982653) / 1.172466510455008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.075173712130733 > 4.163675490114637.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.450013105965308 (T) = (0 -7.1267096432128705) / Math.Sqrt((0.23616896022062805 / (19)) + (0.0008850319647394454 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.8623735218324873 = (3.8266811462184696 - 7.1267096432128705) / 3.8266811462184696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: false)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5343126188653398 > 2.2806500926645814.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.961744846178966 (T) = (0 -3.4983010989832484) / Math.Sqrt((0.033193812127140725 / (20)) + (0.00915181519787125 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6785360520880973 = (2.084138195680316 - 3.4983010989832484) / 2.084138195680316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3397488454288866 > 2.074464391378118.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.216365511392677 (T) = (0 -3.324714856304684) / Math.Sqrt((0.05899022680846609 / (20)) + (0.003949009067632837 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.706260046845744 = (1.9485393580251007 - 3.324714856304684) / 1.9485393580251007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.289220387281342 > 4.417694818984228.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.766225053288288 (T) = (0 -7.151895279240108) / Math.Sqrt((0.20381457489154428 / (20)) + (0.04706578771931088 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7611302938672739 = (4.060968858547785 - 7.151895279240108) / 4.060968858547785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: False)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3419506574345936 > 2.404171030949361.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.733522771470165 (T) = (0 -3.2924816825166707) / Math.Sqrt((0.03615330470352957 / (19)) + (0.08677649914320967 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5468233305726309 = (2.1285441054847554 - 3.2924816825166707) / 2.1285441054847554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.2857509426828715 > 1.3193948419217365.
IsChangePoint: Marked as a change because one of 6/10/2023 2:06:10 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -52.62771731932427 (T) = (0 -4.378310025650239) / Math.Sqrt((0.05835294921006754 / (19)) + (0.004481699194082616 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.735309106925043 = (1.1721412874594797 - 4.378310025650239) / 1.1721412874594797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.726148269332048 > 5.3434511008992045.
IsChangePoint: Marked as a change because one of 5/25/2023 8:00:22 PM, 6/8/2023 3:52:42 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -52.438184836209025 (T) = (0 -7.806984502766608) / Math.Sqrt((0.04367931873522423 / (20)) + (0.005346166320578178 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5739981328982793 = (4.959970624864229 - 7.806984502766608) / 4.959970624864229 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 255)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.172778902639114 > 3.345909933538499.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.196250569139263 (T) = (0 -5.152469912197082) / Math.Sqrt((0.2682929141141859 / (20)) + (0.006464585507404576 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6656571299173567 = (3.0933556610493587 - 5.152469912197082) / 3.0933556610493587 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.400911056703029 > 6.438452145573054.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -61.87534450430256 (T) = (0 -9.356432094020386) / Math.Sqrt((0.041546776504664505 / (20)) + (0.004542179698892284 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5274011150604604 = (6.125720350577342 - 9.356432094020386) / 6.125720350577342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.26 ns 18.17 ns 1.19 0.04 False
6.32 ns 9.50 ns 1.50 0.36 False
10.97 ns 14.56 ns 1.33 0.12 False
6.05 ns 9.64 ns 1.59 0.42 False
10.20 ns 13.11 ns 1.29 0.17 False
3.09 ns 5.28 ns 1.71 0.58 False
4.46 ns 7.80 ns 1.75 0.46 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt32.TryParseHex(value: "FFFFFFFF")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.173296836789056 > 16.00155494515583.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.018549783958363 (T) = (0 -18.204007833929833) / Math.Sqrt((0.29034449010412144 / (20)) + (0.005427345110452224 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2146958862088408 = (14.986473602661107 - 18.204007833929833) / 14.986473602661107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.496182580537974 > 7.116263855339816.
IsChangePoint: Marked as a change because one of 4/18/2023 4:36:42 PM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -34.469609866466676 (T) = (0 -9.49854748958949) / Math.Sqrt((0.12257130943262973 / (19)) + (0.00043904585293518737 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4141980184696508 = (6.716561164375109 - 9.49854748958949) / 6.716561164375109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.559729207661407 > 11.431105877964411.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -38.09315661029341 (T) = (0 -14.468737367973677) / Math.Sqrt((0.06553872416154853 / (20)) + (0.04403624263834053 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3468505813709701 = (10.742644780422363 - 14.468737367973677) / 10.742644780422363 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.637783127062063 > 6.364635699267405.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.12766830267398 (T) = (0 -9.419768508951206) / Math.Sqrt((0.28256572892456644 / (20)) + (0.10717896273425233 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5347015469319463 = (6.137850403410657 - 9.419768508951206) / 6.137850403410657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "3039")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.112455117146983 > 10.5725676879265.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -55.16467670254115 (T) = (0 -13.23593141346115) / Math.Sqrt((0.04873672604085452 / (20)) + (0.0058519344405025195 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.31308374003992656 = (10.080036032628573 - 13.23593141346115) / 10.080036032628573 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.282798743627086 > 3.2285028385727723.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.669995468115655 (T) = (0 -5.283840402217298) / Math.Sqrt((0.2534018689336198 / (19)) + (0.000677440862371662 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.8295168533023431 = (2.8881069844641103 - 5.283840402217298) / 2.8881069844641103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.800008234687074 > 5.180393216396149.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/9/2023 11:04:55 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -58.94088208671362 (T) = (0 -7.832857623766761) / Math.Sqrt((0.04405135467303055 / (20)) + (0.002077433960912929 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6030570424894156 = (4.886200188861014 - 7.832857623766761) / 4.886200188861014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.15 ns 8.40 ns 1.37 0.09 False
15.94 ns 19.21 ns 1.20 0.07 False
1.76 ns 3.52 ns 2.00 0.48 False
2.15 ns 3.93 ns 1.82 0.60 False
3.56 ns 5.62 ns 1.58 0.32 False
13.39 ns 15.87 ns 1.19 0.13 False
3.94 ns 6.20 ns 1.57 0.50 False
11.28 ns 13.32 ns 1.18 0.11 False
5.43 ns 7.43 ns 1.37 0.13 False
3.74 ns 6.38 ns 1.71 0.40 False
4.68 ns 6.70 ns 1.43 0.25 False
1.41 ns 3.13 ns 2.22 0.65 False
1.00 ns 3.83 ns 3.84 0.54 False
1.25 ns 3.50 ns 2.79 0.53 False
3.62 ns 6.03 ns 1.67 0.57 False
0.83 ns 2.71 ns 3.26 0.52 False
4.59 ns 7.41 ns 1.62 0.40 False
8.65 ns 10.83 ns 1.25 0.08 False
1.13 ns 2.67 ns 2.37 0.49 False
14.31 ns 16.73 ns 1.17 0.16 False
3.96 ns 6.05 ns 1.53 0.33 False
0.76 ns 3.78 ns 4.98 0.76 False
11.43 ns 16.98 ns 1.49 0.07 False
3.03 ns 4.67 ns 1.54 0.37 False
4.86 ns 6.76 ns 1.39 0.23 False
4.29 ns 6.11 ns 1.42 0.20 False
2.13 ns 4.26 ns 2.00 0.49 False
0.82 ns 2.23 ns 2.73 0.60 False
18.06 ns 19.86 ns 1.10 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.39946560734132 > 6.446914788227254.
IsChangePoint: Marked as a change because one of 3/28/2023 2:42:21 AM, 4/27/2023 10:53:57 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.657204164825792 (T) = (0 -8.278280510882732) / Math.Sqrt((0.07699563093490075 / (20)) + (0.08617283945783173 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.34370845298713854 = (6.16077132839319 - 8.278280510882732) / 6.16077132839319 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfValue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.20753097642986 > 16.55483875564875.
IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.881678336340606 (T) = (0 -18.89969554449545) / Math.Sqrt((0.17211512657736872 / (17)) + (0.4129636369123182 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.199838578985732 = (15.751865188792365 - 18.89969554449545) / 15.751865188792365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5232886816288627 > 1.676660064887416.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.07737786196303 (T) = (0 -3.382966497519752) / Math.Sqrt((0.16908960834335623 / (20)) + (0.08554572413340167 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.0548014162085835 = (1.6463715037542812 - 3.382966497519752) / 1.6463715037542812 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.StartsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.9254026165189515 > 2.467055586421657.
IsChangePoint: Marked as a change because one of 5/8/2023 9:53:14 PM, 5/11/2023 10:01:37 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.566719947159729 (T) = (0 -3.949663059339918) / Math.Sqrt((0.4622158609238885 / (19)) + (0.000332346372937846 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4267373371676998 = (2.7683182856773247 - 3.949663059339918) / 2.7683182856773247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.616150321670289 > 3.7208951703831494.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 5/9/2023 7:27:03 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.75954007831346 (T) = (0 -5.509586041401633) / Math.Sqrt((0.09419512371267669 / (20)) + (0.08104288579487823 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.4678257913715961 = (3.753569445222278 - 5.509586041401633) / 3.753569445222278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.StartsWith(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.8671439930639 > 14.18318440118886.
IsChangePoint: Marked as a change because one of 6/18/2023 4:50:43 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.444407934567964 (T) = (0 -15.978098945504431) / Math.Sqrt((0.20910205588071956 / (19)) + (0.4568322686996849 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.1708956353094409 = (13.646048771273954 - 15.978098945504431) / 13.646048771273954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.EndsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.204719876830067 > 4.161261750564603.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.093090486279126 (T) = (0 -6.139317316354861) / Math.Sqrt((1.2686769425710773 / (19)) + (0.0535096019320472 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4602253013059147 = (4.204363060182782 - 6.139317316354861) / 4.204363060182782 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.BinarySearch(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.323039122756594 > 12.086595634878563.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.483307868660518 (T) = (0 -13.06398656764709) / Math.Sqrt((0.6405714033695243 / (19)) + (0.06212287025145835 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.13406116404150067 = (11.519649011778537 - 13.06398656764709) / 11.519649011778537 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.BinarySearch(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.433882379900492 > 5.847933406615201.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -90.24904740897895 (T) = (0 -7.4615575379497425) / Math.Sqrt((0.004458808892890508 / (19)) + (0.001693975294356473 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.35882855100929734 = (5.491169237213569 - 7.4615575379497425) / 5.491169237213569 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.38107723831646 > 3.8910187782989367.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -28.025931108272424 (T) = (0 -6.315509335976896) / Math.Sqrt((0.056006386787834886 / (20)) + (0.03933331864676604 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6868657316494775 = (3.7439312551576736 - 6.315509335976896) / 3.7439312551576736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.6980612305943925 > 4.900793498333462.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.989899575574878 (T) = (0 -6.578022131027411) / Math.Sqrt((0.5614660715979126 / (20)) + (0.05813729308018583 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.4676865569157598 = (4.481898468056192 - 6.578022131027411) / 4.481898468056192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceEqual(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.1280356694261835 > 1.632197911824606.
IsChangePoint: Marked as a change because one of 5/8/2023 9:53:14 PM, 5/11/2023 10:01:37 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.716159289179341 (T) = (0 -3.2567118406936286) / Math.Sqrt((0.216818574301767 / (20)) + (0.035423870018285335 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7092077586067014 = (1.9053926149670708 - 3.2567118406936286) / 1.9053926149670708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.EndsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8299762469031053 > 1.2700665638609279.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.249703367426154 (T) = (0 -3.863285441903792) / Math.Sqrt((0.27947181031135643 / (19)) + (0.0005896791848471125 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.5383128536054846 = (1.5219894728170653 - 3.863285441903792) / 1.5219894728170653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4985454986763256 > 1.395431032464973.
IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 3/17/2023 1:20:12 AM, 5/20/2023 3:57:43 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.001608358099407 (T) = (0 -3.4985630483903023) / Math.Sqrt((0.5591545420974912 / (20)) + (0.0010281855048915225 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.3534594805382445 = (1.4865618368709577 - 3.4985630483903023) / 1.4865618368709577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceEqual(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.0270472595759195 > 4.119584869852868.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.784206384470341 (T) = (0 -6.055361902301777) / Math.Sqrt((0.6758015597459339 / (20)) + (0.06771922619465899 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5905604886411405 = (3.807061690231623 - 6.055361902301777) / 3.807061690231623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.713288606987568 > 0.8654170330357202.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -32.65072403922227 (T) = (0 -2.7111476334141105) / Math.Sqrt((0.06380196126118276 / (20)) + (2.862954863099942E-05 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.131299673946714 = (0.8658218362080176 - 2.7111476334141105) / 0.8658218362080176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareTo(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.40730845325062 > 4.740727544749789.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.40461228769543 (T) = (0 -7.379537978244017) / Math.Sqrt((0.47760395301848346 / (20)) + (0.0013121609681531453 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6890082044597944 = (4.369154607276913 - 7.379537978244017) / 4.369154607276913 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.828328082180514 > 9.087452242762078.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.285155656323717 (T) = (0 -10.78660690693132) / Math.Sqrt((0.3353545882766749 / (20)) + (0.0010555497227931968 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.22596699178818855 = (8.7984480652273 - 10.78660690693132) / 8.7984480652273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6650529093321498 > 1.0901403911889649.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/17/2023 6:28:21 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.231941558846916 (T) = (0 -2.6639227519266404) / Math.Sqrt((0.08782130629893019 / (20)) + (0.000772380386359075 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.743347117677816 = (0.9710483718085203 - 2.6639227519266404) / 0.9710483718085203 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.EndsWith(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.730902428646452 > 14.98089303821233.
IsChangePoint: Marked as a change because one of 3/17/2023 7:47:41 PM, 4/25/2023 6:46:56 PM, 5/8/2023 9:53:14 PM, 5/10/2023 10:36:48 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.471365026951636 (T) = (0 -16.473225042912123) / Math.Sqrt((0.25399317594554094 / (19)) + (0.16278256396467927 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.13849389731228834 = (14.469313434003876 - 16.473225042912123) / 14.469313434003876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.051578101009687 > 4.138571505849978.
IsChangePoint: Marked as a change because one of 4/12/2023 2:59:13 PM, 4/28/2023 7:22:08 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -36.253815914315105 (T) = (0 -6.110994953546589) / Math.Sqrt((0.06793632450451714 / (20)) + (0.0007120244999446685 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5405975116231837 = (3.966639506711915 - 6.110994953546589) / 3.966639506711915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.StartsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.779550669038691 > 0.9502753252309923.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.110738018184865 (T) = (0 -3.474579177163494) / Math.Sqrt((0.129252235323151 / (19)) + (0.05172238958298556 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.2207462201240715 = (1.0788118466004577 - 3.474579177163494) / 1.0788118466004577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfValue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.98079439188714 > 15.56689518183571.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.89393317604925 (T) = (0 -16.891779459044024) / Math.Sqrt((0.6466327889623575 / (20)) + (0.013496398207789957 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.14986043027483092 = (14.690286763765476 - 16.891779459044024) / 14.690286763765476 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.665516719491481 > 3.1926774902648116.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.180977270882355 (T) = (0 -5.140776153844728) / Math.Sqrt((0.130373633765113 / (20)) + (0.04466568411263202 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.67133262556543 = (3.0758546056058393 - 5.140776153844728) / 3.0758546056058393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareTo(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.76153973836437 > 5.399157636407854.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/17/2023 2:42:45 PM, 5/29/2023 1:48:21 PM, 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.085474769160522 (T) = (0 -6.497703826665765) / Math.Sqrt((0.033286395792193384 / (17)) + (0.18237870553301552 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.32004851695361036 = (4.9223219777263 - 6.497703826665765) / 4.9223219777263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.11135780999433 > 4.424844081491902.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.051197840592945 (T) = (0 -6.090699904280382) / Math.Sqrt((0.20916167094442642 / (20)) + (0.0012567406723333469 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.40595948306082985 = (4.332059335750334 - 6.090699904280382) / 4.332059335750334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.2600275406219525 > 2.13200687623306.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/4/2023 11:36:00 PM, 5/2/2023 7:50:39 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.412114553669248 (T) = (0 -4.3134288834827395) / Math.Sqrt((0.320961309544846 / (19)) + (0.0005935463489657542 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.9832149032080504 = (2.174967965652806 - 4.3134288834827395) / 2.174967965652806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2340759506183208 > 0.7838056031603163.
IsChangePoint: Marked as a change because one of 4/3/2023 6:15:03 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -27.678438634940953 (T) = (0 -2.2716788384875604) / Math.Sqrt((0.0481017683974093 / (19)) + (0.0011362880461490829 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.720364364723322 = (0.8350641803524008 - 2.2716788384875604) / 0.8350641803524008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.861456261294787 > 19.071310285567044.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.198315189979702 (T) = (0 -19.917653930513968) / Math.Sqrt((0.5403920109875431 / (20)) + (0.045270997972926774 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.08145794427604026 = (18.41740960518574 - 19.917653930513968) / 18.41740960518574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.18 ns 2.77 ns 2.35 0.52 False
8.22 ns 9.71 ns 1.18 0.10 False
4.11 ns 6.08 ns 1.48 0.22 False
3.59 ns 5.92 ns 1.65 0.35 False
2.36 ns 5.11 ns 2.16 0.49 False
1.98 ns 4.24 ns 2.14 0.52 False
3.83 ns 5.04 ns 1.32 0.37 False
1.63 ns 3.44 ns 2.11 0.50 False
6.10 ns 9.61 ns 1.57 0.06 False
4.37 ns 6.83 ns 1.56 0.50 False
4.65 ns 6.96 ns 1.50 0.17 False
11.82 ns 13.30 ns 1.13 0.07 False
3.33 ns 5.53 ns 1.66 0.32 False
2.98 ns 4.02 ns 1.35 0.51 False
4.57 ns 7.63 ns 1.67 0.50 False
2.82 ns 5.04 ns 1.79 0.62 False
4.18 ns 5.49 ns 1.31 0.35 False
4.77 ns 5.87 ns 1.23 0.31 False
9.15 ns 11.66 ns 1.27 0.08 False
0.78 ns 2.38 ns 3.07 0.73 False
5.78 ns 7.78 ns 1.35 0.08 False
1.48 ns 3.58 ns 2.42 0.66 False
4.42 ns 6.91 ns 1.56 0.23 False
1.23 ns 3.38 ns 2.76 0.43 False
1.90 ns 3.46 ns 1.82 0.66 False
5.10 ns 7.51 ns 1.47 0.14 False
0.87 ns 2.83 ns 3.27 0.53 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.766172014054571 > 1.2386298723861673.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -41.07311139985245 (T) = (0 -2.7630030608727334) / Math.Sqrt((0.02997171176551524 / (20)) + (1.6202315615843077E-05 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.3579718073571665 = (1.171771033161558 - 2.7630030608727334) / 1.171771033161558 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.709004506806837 > 8.428402665236403.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.340885570313189 (T) = (0 -10.461672603269704) / Math.Sqrt((0.24084666540228686 / (20)) + (0.13553306184657718 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2642938736165259 = (8.274715888121786 - 10.461672603269704) / 8.274715888121786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.080601994266007 > 4.318401210245283.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.473126670071544 (T) = (0 -5.956248526210785) / Math.Sqrt((0.02903291468665023 / (20)) + (0.08888273763283654 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.44719651326743454 = (4.115715088867202 - 5.956248526210785) / 4.115715088867202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.923190936213385 > 3.99097210715669.
IsChangePoint: Marked as a change because one of 4/7/2023 6:04:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.21882622020433 (T) = (0 -5.852495345790937) / Math.Sqrt((0.018314034027375053 / (19)) + (0.0517372475593387 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5314348261894623 = (3.8215765017915904 - 5.852495345790937) / 3.8215765017915904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.113199271069997 > 2.71973440048491.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.788975138318467 (T) = (0 -5.119864630036319) / Math.Sqrt((0.5732375652451047 / (19)) + (0.011072009750689735 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.8021464539568144 = (2.8409814412114414 - 5.119864630036319) / 2.8409814412114414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.EndsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.239587173516314 > 2.0646479707194256.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 7/28/2023 9:42:28 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.308914006093564 (T) = (0 -4.256381295967806) / Math.Sqrt((0.5021692223003014 / (18)) + (0.0002706282086623078 / (9))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (9) - 2, .025) and -1.0944440924741727 = (2.032224832957814 - 4.256381295967806) / 2.032224832957814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareToDifferent(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.044545545049976 > 4.197690201238512.
IsChangePoint: Marked as a change because one of 4/27/2023 10:53:57 PM, 5/8/2023 6:34:59 AM, 7/20/2023 8:15:02 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.731262944264756 (T) = (0 -5.023949254964893) / Math.Sqrt((0.07528166791970514 / (20)) + (0.0021376338571662996 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3120654508913617 = (3.8290385983045545 - 5.023949254964893) / 3.8290385983045545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4414223743611765 > 1.7375737929862427.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -46.89174299493814 (T) = (0 -3.4276778181763308) / Math.Sqrt((0.023429872934187156 / (19)) + (0.0010775187114357412 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.0387522331549452 = (1.6812625695433523 - 3.4276778181763308) / 1.6812625695433523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareTo(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.608896203291321 > 6.646956232909083.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -27.088875119148373 (T) = (0 -9.68141748201519) / Math.Sqrt((0.2708696619988685 / (20)) + (0.0011396485471669466 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.487156756461116 = (6.510018153737464 - 9.68141748201519) / 6.510018153737464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.EndsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.828121628765504 > 4.166874924448163.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -47.41087579248175 (T) = (0 -6.822097412940181) / Math.Sqrt((0.05404836077069918 / (19)) + (0.0001250261782952899 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5919024308802521 = (4.285499714431531 - 6.822097412940181) / 4.285499714431531 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.95737003092882 > 4.875368404303146.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -43.638239629860344 (T) = (0 -6.93705510481671) / Math.Sqrt((0.0485477625753099 / (19)) + (0.000574251177118808 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.47720151705710895 = (4.696079055372729 - 6.93705510481671) / 4.696079055372729 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.BinarySearch(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.303985772646532 > 12.401434135754192.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.14592227209188 (T) = (0 -13.35235338481156) / Math.Sqrt((0.060160929235985774 / (20)) + (0.02196423342601322 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.13414935507460019 = (11.77301148659851 - 13.35235338481156) / 11.77301148659851 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.531487016558591 > 3.7261772899037173.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -38.4430800379057 (T) = (0 -5.5072029569331775) / Math.Sqrt((0.042117831843683125 / (20)) + (0.003030948483357146 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5425654659186231 = (3.5701583359728253 - 5.5072029569331775) / 3.5701583359728253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.BinarySearch(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.020554154138622 > 3.1052272226868882.
IsChangePoint: Marked as a change because one of 3/6/2023 4:10:52 PM, 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -37.0910482355333 (T) = (0 -4.034261623520791) / Math.Sqrt((0.015697887096064983 / (20)) + (0.0005688660018850621 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.37096239264602665 = (2.9426493718288382 - 4.034261623520791) / 2.9426493718288382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceEqual(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.629921560382171 > 4.814511673194359.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.114465172991856 (T) = (0 -7.6001508616486335) / Math.Sqrt((0.22160503116234653 / (20)) + (0.13402253054732202 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.707584185220569 = (4.450820596389466 - 7.6001508616486335) / 4.450820596389466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.StartsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.0400130881830885 > 2.921828380944245.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.430176762458379 (T) = (0 -4.5601120283462775) / Math.Sqrt((0.07391580152158132 / (20)) + (0.08854418963158649 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7624275887156793 = (2.5874039067155854 - 4.5601120283462775) / 2.5874039067155854 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareToDifferent(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.493845625951311 > 4.595580615991153.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 5/23/2023 1:41:12 PM, 6/16/2023 8:17:59 AM, 7/5/2023 5:05:14 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.517046237201363 (T) = (0 -5.6510328087425625) / Math.Sqrt((0.3354103747355806 / (20)) + (0.007111730040645986 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2896652130293179 = (4.381782769396989 - 5.6510328087425625) / 4.381782769396989 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareToDifferent(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.872008067656676 > 5.021230372968492.
IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.344402574139613 (T) = (0 -5.7570317502990065) / Math.Sqrt((0.1141774276691201 / (20)) + (0.09061084459011533 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2468183095858592 = (4.617378254744471 - 5.7570317502990065) / 4.617378254744471 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.65788370354136 > 9.587946257848635.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 5/19/2023 9:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.830841997684757 (T) = (0 -11.642471716615672) / Math.Sqrt((0.2149809365108722 / (20)) + (0.002487264197503279 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2604688721923449 = (9.236619779721982 - 11.642471716615672) / 9.236619779721982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3808940623488923 > 0.6807865868990245.
IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 5/19/2023 9:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -46.11453210427155 (T) = (0 -2.3813210137792757) / Math.Sqrt((0.023468739441923146 / (19)) + (0.000949577060305237 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.5334819592975837 = (0.6739304293073723 - 2.3813210137792757) / 0.6739304293073723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.BinarySearch(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.7832178156199205 > 5.964757111201717.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 5/6/2023 11:12:53 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.479105716885705 (T) = (0 -7.631480916075704) / Math.Sqrt((0.014414214415332407 / (20)) + (0.06142609378360687 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3310786957013389 = (5.733305581947365 - 7.631480916075704) / 5.733305581947365 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.StartsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5808587946755637 > 1.4388163985659903.
IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -28.475973743010023 (T) = (0 -3.666918588347606) / Math.Sqrt((0.05604422886108456 / (20)) + (0.028573509678322073 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.8115229414974992 = (1.3042463691918154 - 3.666918588347606) / 1.3042463691918154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.90552996202633 > 4.515783869993911.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.556066759224642 (T) = (0 -6.787076276925829) / Math.Sqrt((0.958086131623379 / (20)) + (0.08492161547200132 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.526686268344214 = (4.445626071089795 - 6.787076276925829) / 4.445626071089795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.382351321826838 > 1.29127445342239.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.260841711074525 (T) = (0 -3.368669520537689) / Math.Sqrt((0.7908879792174592 / (19)) + (0.0008389285599632148 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.2814926387356085 = (1.4765200042041724 - 3.368669520537689) / 1.4765200042041724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceEqual(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.462748365393174 > 1.5135324939826311.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.678292208107507 (T) = (0 -3.628288258527063) / Math.Sqrt((0.09278726567214092 / (20)) + (0.06001939262782991 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.4495373073920121 = (1.4812137163936687 - 3.628288258527063) / 1.4812137163936687 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.511702803262452 > 5.5585615766008525.
IsChangePoint: Marked as a change because one of 3/15/2023 4:14:49 PM, 3/16/2023 10:20:48 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -61.84959182729199 (T) = (0 -7.566316944791686) / Math.Sqrt((0.024528019638508335 / (20)) + (0.0006792148989906531 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.4232323816019764 = (5.3162906090396245 - 7.566316944791686) / 5.3162906090396245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8305875476060605 > 0.9044482648976976.
IsChangePoint: Marked as a change because one of 7/26/2023 10:32:59 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -6.019975665533057 (T) = (0 -2.1928443395594726) / Math.Sqrt((0.04724113640176247 / (13)) + (0.6266437322909878 / (14))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (14) - 2, .025) and -1.5247795059782177 = (0.8685290475335438 - 2.1928443395594726) / 0.8685290475335438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.86 ns 22.55 ns 1.20 0.04 False
6.62 ns 9.74 ns 1.47 0.37 False
16.27 ns 21.08 ns 1.30 0.10 False
3.91 ns 5.60 ns 1.43 0.60 False
5.97 ns 9.58 ns 1.60 0.35 False
17.32 ns 20.90 ns 1.21 0.05 False
18.92 ns 22.63 ns 1.20 0.04 False
10.32 ns 14.53 ns 1.41 0.07 False
17.28 ns 21.05 ns 1.22 0.02 False
10.74 ns 14.60 ns 1.36 0.08 False
5.91 ns 9.69 ns 1.64 0.35 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

System.Tests.Perf_Int32.TryParse(value: "-2147483648")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.548965545308523 > 19.756934932163464.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.890499527389753 (T) = (0 -22.70661844238206) / Math.Sqrt((0.045663798694832225 / (20)) + (0.2585609416241164 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.20989219244480514 = (18.767472493974232 - 22.70661844238206) / 18.767472493974232 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryFormat(value: 2147483647)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.742615945499761 > 7.438836103873657.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.309064081999367 (T) = (0 -9.692756353488134) / Math.Sqrt((0.12591621277348422 / (19)) + (0.04962775849130691 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3690445692271106 = (7.07994215189075 - 9.692756353488134) / 7.07994215189075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryFormat(value: -2147483648)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.078795571286758 > 17.668949743222495.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -32.40983780770047 (T) = (0 -20.764085245831847) / Math.Sqrt((0.09702400953633115 / (19)) + (0.06526850055530621 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.23077246905725157 = (16.870774873390484 - 20.764085245831847) / 16.870774873390484 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.60041812612832 > 4.104854303635711.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.18300268655641 (T) = (0 -5.62650615849852) / Math.Sqrt((0.6420702804609246 / (19)) + (0.00031051458918169493 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4990645690531382 = (3.7533447688997272 - 5.62650615849852) / 3.7533447688997272 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.576795197568886 > 6.432975002028186.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.58434099741109 (T) = (0 -9.613361653710879) / Math.Sqrt((0.8507271504405494 / (20)) + (0.0035198811849984023 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.557376469233898 = (6.172792413153556 - 9.613361653710879) / 6.172792413153556 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.900457021358307 > 18.17575400094927.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -51.43081099411366 (T) = (0 -20.961694234745355) / Math.Sqrt((0.08173486650027718 / (20)) + (0.007244889336721668 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.21299192307544831 = (17.28098418132792 - 20.961694234745355) / 17.28098418132792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.63005778558423 > 19.696592344673167.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.926920246958048 (T) = (0 -22.521758576721858) / Math.Sqrt((0.05470180554112143 / (19)) + (0.09808912320570821 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.20875427929110335 = (18.63220586894647 - 22.521758576721858) / 18.63220586894647 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.526834495092128 > 11.39521267695524.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -96.34917270148138 (T) = (0 -14.552736435302725) / Math.Sqrt((0.02554423183092068 / (19)) + (0.0012100470590182633 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.34751359395273673 = (10.799695454362261 - 14.552736435302725) / 10.799695454362261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "2147483647")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.05028059489367 > 18.16544290695939.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -70.43486460425223 (T) = (0 -21.034939606826345) / Math.Sqrt((0.051610232651764384 / (20)) + (0.003060520229282561 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.22540666736761136 = (17.16568072215005 - 21.034939606826345) / 17.16568072215005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.601413976593543 > 11.310946884279508.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.531569747842315 (T) = (0 -14.383567808672266) / Math.Sqrt((0.08480079084526117 / (20)) + (0.13638161085941167 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.33686697232953955 = (10.75916161172594 - 14.383567808672266) / 10.75916161172594 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.690828645822183 > 6.384202648228123.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -71.86186761903606 (T) = (0 -9.648722795549633) / Math.Sqrt((0.044709634477000826 / (19)) + (0.0013367188751712963 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6016928881636816 = (6.02407794081658 - 9.648722795549633) / 6.02407794081658 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.00 ns 2.31 ns ∞ 0.77 False
0.00 ns 2.01 ns ∞ 0.76 False
4.29 ns 6.50 ns 1.51 0.48 False
2.91 ns 6.33 ns 2.17 0.45 False
10.57 ns 13.05 ns 1.23 0.22 False
3.12 ns 4.55 ns 1.46 0.32 False
12.59 ns 16.33 ns 1.30 0.03 False
0.41 ns 2.33 ns 5.67 0.73 False
4.29 ns 5.69 ns 1.33 0.32 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.Primitives.StringSegmentBenchmark.EndsWith

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3145790994613784 > 0.48580092736214175.
IsChangePoint: Marked as a change because one of 3/11/2023 4:36:40 PM, 3/13/2023 3:41:35 AM, 3/25/2023 12:18:49 AM, 3/29/2023 12:02:44 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -41.86944275440094 (T) = (0 -2.2909745748228034) / Math.Sqrt((0.035026109755719737 / (19)) + (0.0014892244764623805 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -4.838685660668479 = (0.3923784748775988 - 2.2909745748228034) / 0.3923784748775988 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.StartsWith

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.013744311776993 > 0.3645342566376365.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 5/4/2023 12:54:08 PM, 5/11/2023 10:01:37 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.298948384755432 (T) = (0 -2.0671165703794565) / Math.Sqrt((0.3089251620925155 / (19)) + (0.001548097073049555 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -3.2339181444821135 = (0.4882278069247612 - 2.0671165703794565) / 0.4882278069247612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.497297290805108 > 4.525942054221353.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 5/3/2023 7:05:50 AM, 5/23/2023 1:41:12 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.783842449638106 (T) = (0 -6.687284984975567) / Math.Sqrt((0.38639513035403494 / (20)) + (0.1106626220710556 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5583522395416318 = (4.291253809820648 - 6.687284984975567) / 4.291253809820648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.330593494629517 > 2.64649302732685.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/3/2023 7:05:50 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.094650296464872 (T) = (0 -6.294302461721825) / Math.Sqrt((0.9905744155590293 / (19)) + (0.046119096670149164 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.0167111260953154 = (3.1210729093901666 - 6.294302461721825) / 3.1210729093901666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.046770054105211 > 11.086950213834065.
IsChangePoint: Marked as a change because one of 5/10/2023 2:36:50 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.365562113191913 (T) = (0 -13.012045261555778) / Math.Sqrt((0.05220625659632151 / (20)) + (0.06075488072500418 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.24837997925508576 = (10.423144777858523 - 13.012045261555778) / 10.423144777858523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.552499355378959 > 3.5259235309526695.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 6/16/2023 10:57:50 PM, 7/5/2023 5:05:14 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.359837390714505 (T) = (0 -4.5894972656565) / Math.Sqrt((0.5831944758706876 / (19)) + (0.002681749906537244 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4729963584507829 = (3.1157560161815216 - 4.5894972656565) / 3.1157560161815216 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.32568480207741 > 13.11743205134898.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.652317375593846 (T) = (0 -16.082325274948506) / Math.Sqrt((0.011837533966428115 / (16)) + (0.4005173442541823 / (10))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (10) - 2, .025) and -0.2848283900155938 = (12.517099871021154 - 16.082325274948506) / 12.517099871021154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_String

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3315851127386416 > 0.4656703200932771.
IsChangePoint: Marked as a change because one of 4/7/2023 6:04:01 AM, 5/3/2023 1:18:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -36.194490106462396 (T) = (0 -2.3912645206974603) / Math.Sqrt((0.044872769275045674 / (19)) + (0.006361263784090268 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -6.440820734454714 = (0.321371069944571 - 2.3912645206974603) / 0.321371069944571 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.689275949590251 > 3.7554563474245115.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.144628996901105 (T) = (0 -5.6921219737569775) / Math.Sqrt((0.04293653530862882 / (19)) + (0.021172263318672894 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5928789809381888 = (3.573480497811816 - 5.6921219737569775) / 3.573480497811816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.21 ns 15.05 ns 1.34 0.05 False
5.84 ns 7.40 ns 1.27 0.39 False
22.59 ns 24.98 ns 1.11 0.09 False
11.15 ns 15.01 ns 1.35 0.07 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

System.Tests.Perf_Int64.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.050597081614082 > 11.765408356560739.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.792445680578332 (T) = (0 -14.860979083524825) / Math.Sqrt((0.026099383761106244 / (20)) + (0.10167987990204368 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.33732163977975904 = (11.112494288189527 - 14.860979083524825) / 11.112494288189527 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.400047328754399 > 6.172380245346144.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -36.56141116989971 (T) = (0 -7.363700353289765) / Math.Sqrt((0.03599356258107922 / (20)) + (0.0004216764999722708 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.27247204218335863 = (5.786925063323853 - 7.363700353289765) / 5.786925063323853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.98152124920906 > 23.617299018607678.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.653748158903255 (T) = (0 -24.861799138131172) / Math.Sqrt((0.027570397825735227 / (20)) + (0.1055367672271338 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.1064964620345862 = (22.468936857164625 - 24.861799138131172) / 22.468936857164625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.00880499555198 > 11.724157415225106.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -28.50564266013522 (T) = (0 -14.83851006576799) / Math.Sqrt((0.07155715854114222 / (20)) + (0.0943941954417325 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.33500146666294284 = (11.11497660213011 - 14.83851006576799) / 11.11497660213011 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.44 ns 9.14 ns 1.42 0.34 False
1.68 ns 6.86 ns 4.09 0.33 False
5.85 ns 9.55 ns 1.63 0.33 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'

Payloads

Baseline
Compare

System.Tests.Perf_Boolean.TryParse(value: " True ")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.142634622762891 > 5.824872983299459.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -56.38445221923017 (T) = (0 -9.13715496137319) / Math.Sqrt((0.0779166277054118 / (19)) + (0.00047122706750604487 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6622415724715193 = (5.496887523867861 - 9.13715496137319) / 5.496887523867861 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.TryParse(value: "Bogus")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.85699116391274 > 3.9022105686167885.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.10894248890041 (T) = (0 -6.898321486349292) / Math.Sqrt((0.24870263843883497 / (20)) + (0.0010615842213473328 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.8990224426527513 = (3.632564487607109 - 6.898321486349292) / 3.632564487607109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.Parse(value: " True ")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.549066684940643 > 6.324975388158662.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -43.01403864148007 (T) = (0 -9.446958118222854) / Math.Sqrt((0.10839664430355961 / (19)) + (0.0037509444202845897 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5617732722427233 = (6.048866558368565 - 9.446958118222854) / 6.048866558368565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.89 ns 12.75 ns 1.43 0.08 False
9.14 ns 12.91 ns 1.41 0.10 False
6.21 ns 9.62 ns 1.55 0.31 False
6.07 ns 9.69 ns 1.60 0.35 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

System.Tests.Perf_Byte.Parse(value: "255")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.754857090497849 > 9.719461092449079.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.370643316343738 (T) = (0 -12.69267546384152) / Math.Sqrt((0.04559011954744424 / (20)) + (0.11134723627082758 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.36903142164741226 = (9.271281333022946 - 12.69267546384152) / 9.271281333022946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.TryParse(value: "255")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.910921774625956 > 9.579073751576953.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -43.672875846230966 (T) = (0 -12.824760399538965) / Math.Sqrt((0.029565330168678162 / (20)) + (0.04102391404237482 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.4118838810856305 = (9.083438497560937 - 12.824760399538965) / 9.083438497560937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.62003836926298 > 6.467257526146121.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -33.95969368175044 (T) = (0 -9.663805064734115) / Math.Sqrt((0.21605093796338712 / (20)) + (0.005736015318314867 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6098907044681029 = (6.002770894889396 - 9.663805064734115) / 6.002770894889396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.687718076930553 > 6.317587179241568.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.495126851959373 (T) = (0 -9.621031903060487) / Math.Sqrt((0.28979798073367324 / (20)) + (0.0022667669273241696 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5952282907377567 = (6.031131693765899 - 9.621031903060487) / 6.031131693765899 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.15 ns 9.86 ns 1.38 0.43 False
7.13 ns 10.76 ns 1.51 0.45 False
13.89 ns 17.65 ns 1.27 0.10 False
6.56 ns 9.10 ns 1.39 0.44 False
7.27 ns 9.07 ns 1.25 0.53 False
6.20 ns 9.16 ns 1.48 0.38 False
7.35 ns 9.93 ns 1.35 0.35 False
7.38 ns 9.87 ns 1.34 0.37 False
6.14 ns 9.09 ns 1.48 0.39 False
6.21 ns 9.94 ns 1.60 0.42 False
7.32 ns 9.51 ns 1.30 0.36 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.862120829710683 > 7.508278906644375.
IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 5/23/2023 1:41:12 PM, 6/16/2023 8:17:59 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.825110891409576 (T) = (0 -9.962392416771037) / Math.Sqrt((0.059612651000763926 / (20)) + (0.05249495708442848 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3805465114429824 = (7.216267133483312 - 9.962392416771037) / 7.216267133483312 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.764721243905509 > 7.513062013571816.
IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.66602648148461 (T) = (0 -10.83405788404933) / Math.Sqrt((0.1425467261526133 / (19)) + (0.04448799089569439 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.47567034847945294 = (7.34178734106358 - 10.83405788404933) / 7.34178734106358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.64832859951934 > 16.580657225280394.
IsChangePoint: Marked as a change because one of 4/27/2023 10:53:57 PM, 5/3/2023 7:05:50 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.90912194839287 (T) = (0 -17.543051032671393) / Math.Sqrt((0.2860522608580921 / (20)) + (0.038872666205792133 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.12577373354968388 = (15.583105654238613 - 17.543051032671393) / 15.583105654238613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.100183337714501 > 6.884730776993656.
IsChangePoint: Marked as a change because one of 3/18/2023 3:39:51 AM, 6/16/2023 8:17:59 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.84188948498841 (T) = (0 -9.2117949714073) / Math.Sqrt((0.09252634690656011 / (20)) + (0.04033427746547856 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3790467579271565 = (6.679827872735467 - 9.2117949714073) / 6.679827872735467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.072547119982802 > 7.590196192448673.
IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.61791364219793 (T) = (0 -9.209286479491931) / Math.Sqrt((0.13257033989694952 / (20)) + (0.04287206042826003 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.29584648693772136 = (7.1067727329761485 - 9.209286479491931) / 7.1067727329761485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.15995725715556 > 6.966793777387001.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -28.067551431241956 (T) = (0 -9.140496265821096) / Math.Sqrt((0.07727353866147206 / (19)) + (0.03382342024111192 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.40779704576245573 = (6.4927656250838695 - 9.140496265821096) / 6.4927656250838695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.932115310047056 > 7.741517751030168.
IsChangePoint: Marked as a change because one of 5/2/2023 7:50:39 PM, 5/19/2023 1:23:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -30.687957523927253 (T) = (0 -9.995986567474148) / Math.Sqrt((0.04848968738699786 / (20)) + (0.03294860800397641 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3499959819950897 = (7.40445653230878 - 9.995986567474148) / 7.40445653230878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.869556134814738 > 7.811162853386785.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.70514327657332 (T) = (0 -9.852698316627938) / Math.Sqrt((0.015042368458940041 / (19)) + (0.07294334176463244 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3227751967087453 = (7.44850549144168 - 9.852698316627938) / 7.44850549144168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreNonSpace, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.70804432578676 (T) = (0 -9.093219495562797) / Math.Sqrt((0.07718687677059519 / (13)) + (0.09944136213939582 / (7))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (7) - 2, .025) and -0.3819557235716466 = (6.579964423217185 - 9.093219495562797) / 6.579964423217185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.944504214778435 > 6.444014079233997.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -41.94875918992476 (T) = (0 -9.592355862353784) / Math.Sqrt((0.05896752640264097 / (20)) + (0.02505493691429497 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.546367934774928 = (6.2031523330506335 - 9.592355862353784) / 6.2031523330506335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.512646461259271 > 7.67238977130053.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/14/2023 7:13:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.334600920512806 (T) = (0 -9.239180376767319) / Math.Sqrt((0.22965639218648493 / (19)) + (0.015115746756640314 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3103769235193687 = (7.050780741737287 - 9.239180376767319) / 7.050780741737287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.81 ns 9.72 ns 1.67 0.37 False
12.08 ns 15.82 ns 1.31 0.07 False
10.18 ns 15.70 ns 1.54 0.07 False
13.19 ns 17.15 ns 1.30 0.08 False
6.34 ns 9.63 ns 1.52 0.36 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

System.Tests.Perf_Int16.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.721749031343094 > 6.238643459666231.
IsChangePoint: Marked as a change because one of 5/13/2023 2:42:33 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -59.600730044658505 (T) = (0 -9.672016287925175) / Math.Sqrt((0.07354262943517341 / (19)) + (0.0019198587924925809 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6576278784305924 = (5.834853777364337 - 9.672016287925175) / 5.834853777364337 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "32767")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.817210251543141 > 12.682443024840383.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -54.23850277827957 (T) = (0 -15.77121309109101) / Math.Sqrt((0.06317484440978546 / (19)) + (0.003033544567774104 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.267161193904061 = (12.446098544495891 - 15.77121309109101) / 12.446098544495891 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "32767")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.704091696134688 > 12.626049362906619.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -30.88651724267186 (T) = (0 -15.797190142809216) / Math.Sqrt((0.26657013432749455 / (19)) + (0.005531676313400388 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.31237357308588576 = (12.0371138727398 - 15.797190142809216) / 12.0371138727398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "-32768")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.146376300191204 > 13.990356296872378.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -106.41409488723308 (T) = (0 -17.07452423031918) / Math.Sqrt((0.012874832312208814 / (19)) + (0.00408779849923269 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.28429997389354056 = (13.294810073502765 - 17.07452423031918) / 13.294810073502765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.630235099032172 > 6.537516637025731.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.012271011688647 (T) = (0 -9.694813617264272) / Math.Sqrt((0.35329729801965404 / (19)) + (0.004771370389793127 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5241073591296691 = (6.3609781549774995 - 9.694813617264272) / 6.3609781549774995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.03 ns 1.29 ns 41.71 0.92 False
4.04 ns 6.33 ns 1.57 0.19 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector3.EqualsBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2872041008771589 > 0.04351357135991991.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -48.62437398138524 (T) = (0 -1.2730572312342325) / Math.Sqrt((0.010842734729975734 / (20)) + (0.0004890605316134155 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -17.14656571388686 = (0.07015416863478534 - 1.2730572312342325) / 0.07015416863478534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.3259052055265945 > 4.240812320821547.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -98.05507116530859 (T) = (0 -6.285191699636129) / Math.Sqrt((0.005175537080895373 / (20)) + (0.0019536207246044547 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.566947475698164 = (4.0111055393453565 - 6.285191699636129) / 4.0111055393453565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.43 ns 12.77 ns 1.22 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.771542963659313 > 10.92890839642986.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.403243302008068 (T) = (0 -13.004550968514868) / Math.Sqrt((0.007223497565092411 / (19)) + (0.07724078146466673 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.2507584883400016 = (10.397331770879623 - 13.004550968514868) / 10.397331770879623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.37 ns 7.98 ns 1.25 0.39 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix4x4.EqualsBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.97823378089463 > 6.595639772658973.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.193224662933654 (T) = (0 -7.730197941415436) / Math.Sqrt((0.022181856217846734 / (20)) + (0.11935610573748634 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2424353252564801 = (6.2218111351749155 - 7.730197941415436) / 6.2218111351749155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.90 ns 1.98 ns 2.20 0.83 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 143.24.20.36)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9763356801446355 > 0.9450979645232976.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.876406303766586 (T) = (0 -1.9276919536020307) / Math.Sqrt((0.10333051607423764 / (20)) + (0.003294458005906926 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.6203042848995988 = (0.7356748468912623 - 1.9276919536020307) / 0.7356748468912623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.06 ns 10.31 ns 1.28 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

System.Tests.Perf_Array.IndexOfChar

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.307998113002046 > 8.445153588664889.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/1/2023 5:53:03 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.088587029506217 (T) = (0 -10.072693007393692) / Math.Sqrt((0.013911224871263946 / (20)) + (0.13966952276402753 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.25154159323462455 = (8.048228729946317 - 10.072693007393692) / 8.048228729946317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.55 ns 21.07 ns 1.08 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.06713216892234 > 20.54569961657051.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -38.83320458270791 (T) = (0 -20.999835815426387) / Math.Sqrt((0.005608859241110383 / (20)) + (0.007955847929279383 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.07481799426839415 = (19.538038930694057 - 20.999835815426387) / 19.538038930694057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.62 ns 14.45 ns 1.36 0.07 False
9.51 ns 13.32 ns 1.40 0.10 False
10.87 ns 14.45 ns 1.33 0.08 False
9.76 ns 13.49 ns 1.38 0.07 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

System.Tests.Perf_SByte.TryParse(value: "-128")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.45120383750338 > 11.146772711455808.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -39.20073954332818 (T) = (0 -14.345005101517716) / Math.Sqrt((0.040950527174706715 / (20)) + (0.051693538470654554 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.36128713888825786 = (10.537824601232227 - 14.345005101517716) / 10.537824601232227 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "127")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.319063822391403 > 10.21845655814252.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -73.32553185661129 (T) = (0 -13.284858165935495) / Math.Sqrt((0.04268514301029814 / (20)) + (0.00077708132484472 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.35416324619280287 = (9.810381579389007 - 13.284858165935495) / 9.810381579389007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "-128")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.45422550450519 > 11.155430099596233.
IsChangePoint: Marked as a change because one of 4/19/2023 4:06:12 AM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -33.66279467874756 (T) = (0 -14.42246898216271) / Math.Sqrt((0.2436808801831567 / (20)) + (0.0018559266216166264 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.35212592899610756 = (10.666513135260073 - 14.42246898216271) / 10.666513135260073 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.TryParse(value: "127")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.493096475636161 > 10.2625522999314.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -69.76949795541712 (T) = (0 -13.44005802001122) / Math.Sqrt((0.05572229925567767 / (20)) + (0.001659942842106332 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.39944539363387893 = (9.603845981522728 - 13.44005802001122) / 9.603845981522728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.83 ns 9.65 ns 1.66 0.37 False
6.30 ns 9.73 ns 1.54 0.35 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt16.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.653000154754535 > 6.2392680906965845.
IsChangePoint: Marked as a change because one of 5/13/2023 2:42:33 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -27.99784501157875 (T) = (0 -9.62217890279474) / Math.Sqrt((0.34977252910393014 / (20)) + (0.001038300620719394 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6297873311906775 = (5.903947538827071 - 9.62217890279474) / 5.903947538827071 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.73239412641856 > 6.595446974675298.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.298422997046917 (T) = (0 -9.694279802178567) / Math.Sqrt((0.31630877136753793 / (20)) + (0.0004909770876074389 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5195560253953055 = (6.379679090579529 - 9.694279802178567) / 6.379679090579529 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.20 ns 6.43 ns 1.53 0.23 False
3.79 ns 6.73 ns 1.77 0.25 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

System.Tests.Perf_Random.Next_int_int

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.426101817233614 > 4.361326771716456.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.42156540288789 (T) = (0 -6.520546851529865) / Math.Sqrt((0.03687159929698829 / (19)) + (0.0691362996876881 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5555216554108269 = (4.191871472086791 - 6.520546851529865) / 4.191871472086791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Random.Next_int

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.733340673668392 > 4.145477396398924.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.525013846900613 (T) = (0 -6.659390941930746) / Math.Sqrt((1.2754135027524063 / (19)) + (0.08595188004714222 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6751782010571489 = (3.975332855769152 - 6.659390941930746) / 3.975332855769152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
99.93 ms 135.19 ms 1.35 0.41 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_FileStream.ReadAsync(fileSize: 104857600, userBufferSize: 4096, options: None)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/17/2023 10:06:24 AM, 6/1/2023 5:52:46 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -4.94727535414699 (T) = (0 -132329752.87385128) / Math.Sqrt((72673914044368.22 / (13)) + (27247162403804.547 / (7))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (7) - 2, .025) and -0.13010520539310538 = (117095074.19516803 - 132329752.87385128) / 117095074.19516803 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.92 ns 5.12 ns 1.75 0.37 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector2.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.116298875197848 > 3.0660432165875653.
IsChangePoint: Marked as a change because one of 4/27/2023 10:53:57 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -142.6621335217523 (T) = (0 -5.118758903123054) / Math.Sqrt((0.004590377788834407 / (20)) + (8.307122305829401E-05 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7637067914234495 = (2.9022731714900383 - 5.118758903123054) / 2.9022731714900383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.30 ns 6.80 ns 1.28 0.36 False
10.76 ns 14.98 ns 1.39 0.06 False
5.09 ns 7.77 ns 1.53 0.46 False
11.29 ns 13.80 ns 1.22 0.09 False
5.93 ns 9.50 ns 1.60 0.36 False
22.83 ns 25.58 ns 1.12 0.07 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt64.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.799066569202825 > 5.6005638632341075.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -34.15227754763023 (T) = (0 -6.8301281949769885) / Math.Sqrt((0.04078099795643542 / (20)) + (0.0004304492892703749 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2973018449065538 = (5.264872027888753 - 6.8301281949769885) / 5.264872027888753 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.981467771578778 > 11.78061800182219.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -60.563773641083046 (T) = (0 -15.005301153313154) / Math.Sqrt((0.07534456846739392 / (19)) + (0.0007908970231971634 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3472756298038177 = (11.137513973661155 - 15.005301153313154) / 11.137513973661155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParseHex(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.774505455571249 > 4.8940733190913335.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.195143043428054 (T) = (0 -7.680131643043347) / Math.Sqrt((0.03713384929038235 / (19)) + (0.10611593326947981 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6078770668095013 = (4.776566443778551 - 7.680131643043347) / 4.776566443778551 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParseHex(value: "3039")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.797708734877059 > 11.614739944085489.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -58.31653226034207 (T) = (0 -13.712962557248714) / Math.Sqrt((0.03292078202139901 / (19)) + (0.002989456439693765 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.24631350991776238 = (11.002819473692105 - 13.712962557248714) / 11.002819473692105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.503617450725057 > 6.23011957386903.
IsChangePoint: Marked as a change because one of 5/13/2023 2:36:22 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -82.99587585237806 (T) = (0 -9.534042909976561) / Math.Sqrt((0.029005741656231376 / (19)) + (0.002637061437130564 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6123438991974419 = (5.913157183602216 - 9.534042909976561) / 5.913157183602216 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.58185976006165 > 23.97475575681471.
IsChangePoint: Marked as a change because one of 3/9/2023 9:18:02 PM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.46572973701927 (T) = (0 -25.416608130339487) / Math.Sqrt((0.15697930904316185 / (20)) + (0.12035650305463996 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.11422973131236355 = (22.8109225737526 - 25.416608130339487) / 22.8109225737526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_SpinLock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.92 ns 6.72 ns 3.49 0.58 True
13.48 ns 15.22 ns 1.13 0.09 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_SpinLock*'

Payloads

Baseline
Compare

System.Threading.Tests.Perf_SpinLock.TryEnter_Fail

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.724045254553717 > 2.0067298105426934.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -33.491504908879406 (T) = (0 -6.610576414717622) / Math.Sqrt((0.22799265797700097 / (20)) + (0.06265336562136424 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.606650659307494 = (1.8328851444644507 - 6.610576414717622) / 1.8328851444644507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Threading.Tests.Perf_SpinLock.TryEnterExit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.222964079608165 > 14.037876206892829.
IsChangePoint: Marked as a change because one of 5/20/2023 7:49:01 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.625687816383119 (T) = (0 -14.952767938072949) / Math.Sqrt((0.03774463510313022 / (20)) + (0.11319151296885871 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.11666727000915118 = (13.390531217011858 - 14.952767938072949) / 13.390531217011858 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.49 ns 7.66 ns 1.70 0.52 False
4.15 ns 6.91 ns 1.67 0.49 False
1.26 ns 2.55 ns 2.03 0.75 False
19.29 ns 22.77 ns 1.18 0.09 False
7.24 ns 10.15 ns 1.40 0.41 False
3.43 ns 6.16 ns 1.80 0.51 False
6.80 ns 10.17 ns 1.50 0.51 False
2.36 ns 5.41 ns 2.29 0.45 False
0.86 ns 2.64 ns 3.07 0.74 False
4.24 ns 7.41 ns 1.75 0.52 False
8.43 ns 12.05 ns 1.43 0.30 False
5.14 ns 8.84 ns 1.72 0.39 False
6.90 ns 9.41 ns 1.36 0.48 False
3.66 ns 6.50 ns 1.77 0.45 False
3.57 ns 6.41 ns 1.80 0.37 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.663487714725632 > 4.504041165036079.
IsChangePoint: Marked as a change because one of 5/23/2023 1:41:12 PM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -65.47185600587778 (T) = (0 -7.644979231459248) / Math.Sqrt((0.0455602923947159 / (19)) + (0.0017056461441171489 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7862218830991845 = (4.279971768230062 - 7.644979231459248) / 4.279971768230062 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.905189950347727 > 4.45220586066706.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -33.693891450291986 (T) = (0 -7.39745632870721) / Math.Sqrt((0.04582856709335872 / (20)) + (0.047590464750183976 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7676040980363891 = (4.185018770280607 - 7.39745632870721) / 4.185018770280607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.IsValid_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.55377121221433 > 1.2445731250541163.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.65973629373977 (T) = (0 -2.6006737512794595) / Math.Sqrt((0.07831774460886316 / (19)) + (0.0012841125303871082 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.20498265240433 = (1.179453157349635 - 2.6006737512794595) / 1.179453157349635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Chars(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.76945262022397 > 20.056158122633494.
IsChangePoint: Marked as a change because one of 5/11/2023 10:01:37 AM, 7/14/2023 7:13:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -39.71022002922264 (T) = (0 -22.8742619154937) / Math.Sqrt((0.06933150171566221 / (19)) + (0.033467279579646333 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.18961730684678016 = (19.228252467278413 - 22.8742619154937) / 19.228252467278413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.14543392830195 > 7.577400464291017.
IsChangePoint: Marked as a change because one of 5/21/2023 3:58:09 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.921833606451045 (T) = (0 -9.991256120509052) / Math.Sqrt((0.22052973029752362 / (20)) + (0.06302691619658755 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3661805764638726 = (7.313276365244285 - 9.991256120509052) / 7.313276365244285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Bytes(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.164673216211706 > 3.5313154688104285.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -60.046481294893674 (T) = (0 -6.169097779568031) / Math.Sqrt((0.04108808499423246 / (19)) + (0.00028209977938323277 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.8410508674788801 = (3.3508567788873442 - 6.169097779568031) / 3.3508567788873442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Bytes_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.173289965244582 > 7.361108853562152.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.215237392213837 (T) = (0 -10.023494906671147) / Math.Sqrt((0.4254472829136762 / (20)) + (0.0859159683477898 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.4589341597179187 = (6.870423068721048 - 10.023494906671147) / 6.870423068721048 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.IsValid_Bytes(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.405077337121696 > 2.4156788180527067.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.930776704156813 (T) = (0 -5.375611279045186) / Math.Sqrt((0.4789332575682885 / (20)) + (0.00039103458433143384 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.2006718555538933 = (2.4427136946740218 - 5.375611279045186) / 2.4427136946740218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.IsValid_Bytes(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.6396716361588886 > 1.1857409487817747.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -38.790121547427134 (T) = (0 -2.6372242515324937) / Math.Sqrt((0.03192306083404084 / (20)) + (0.00014243617065969193 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.4472243624552144 = (1.0776389333124565 - 2.6372242515324937) / 1.0776389333124565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.409350400491973 > 4.440743168330651.
IsChangePoint: Marked as a change because one of 5/22/2023 11:33:11 PM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -63.58059980925278 (T) = (0 -7.357459578124308) / Math.Sqrt((0.047195352416969504 / (20)) + (0.0016299366431173183 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.7857567183066136 = (4.120079461384412 - 7.357459578124308) / 4.120079461384412 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Bytes(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.050390023112481 > 9.265932705421362.
IsChangePoint: Marked as a change because one of 7/5/2023 5:05:14 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.28889079590068 (T) = (0 -12.188511866142715) / Math.Sqrt((0.23371512608671674 / (19)) + (0.07578444043468631 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3616957899338559 = (8.950980061952581 - 12.188511866142715) / 8.950980061952581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.IsValid_Chars(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.83861857073876 > 5.807915835093704.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.70191571691075 (T) = (0 -8.852014676974436) / Math.Sqrt((0.29806018750870705 / (19)) + (0.0002724657605256873 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5726074699068158 = (5.6288774194230164 - 8.852014676974436) / 5.6288774194230164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Bytes(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.407065532484706 > 7.29052713094927.
IsChangePoint: Marked as a change because one of 5/23/2023 1:41:12 PM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.571229298267703 (T) = (0 -9.29581617592758) / Math.Sqrt((0.03459439906070193 / (20)) + (0.05680620776605136 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.35550528556875555 = (6.857823628498177 - 9.29581617592758) / 6.857823628498177 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.499723813933275 > 3.828623568041417.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -63.10395125041196 (T) = (0 -6.477551424473051) / Math.Sqrt((0.03958200271902532 / (19)) + (0.00031112633854008166 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.8160941089548832 = (3.5667487673316227 - 6.477551424473051) / 3.5667487673316227 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Bytes_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.408837905667618 > 3.756606106020682.
IsChangePoint: Marked as a change because one of 6/21/2023 5:18:32 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.675481221814366 (T) = (0 -6.472582857981638) / Math.Sqrt((0.16761807511913343 / (19)) + (0.0017976912934121218 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7757335749952241 = (3.645019134133929 - 6.472582857981638) / 3.645019134133929 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.82 ns 18.96 ns 1.06 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.962222421774527 > 18.83647608234013.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.4019586391665 (T) = (0 -19.303393000965823) / Math.Sqrt((0.011197023746652303 / (19)) + (0.041031429033384796 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.07805819692733262 = (17.90570588488089 - 19.303393000965823) / 17.90570588488089 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.10 ns 2.29 ns 22.21 0.87 False
0.77 ns 1.92 ns 2.49 0.75 False
0.71 ns 1.91 ns 2.71 0.66 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix3x2.InvertBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.290598944248375 > 0.09726183872389554.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.201608764221424 (T) = (0 -2.2076226856623076) / Math.Sqrt((0.5906725389883491 / (19)) + (0.11798243495529187 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -4.360730462935197 = (0.41181378189522944 - 2.2076226856623076) / 0.41181378189522944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualityOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9215030238356692 > 0.7338490597799042.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.28420772552931 (T) = (0 -1.98670348682764) / Math.Sqrt((0.12055780722623437 / (20)) + (0.15626743116955802 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.3570013292774057 = (0.591808966383483 - 1.98670348682764) / 0.591808966383483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.InequalityOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9119688293092296 > 0.7348001371475352.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.67477555243777 (T) = (0 -2.1576579058350585) / Math.Sqrt((0.09290522160853917 / (20)) + (0.09313724672385894 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.635640033474103 = (0.5934740199714625 - 2.1576579058350585) / 0.5934740199714625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
21.47 ns 25.37 ns 1.18 0.03 False
6.23 ns 8.11 ns 1.30 0.05 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.36556227329811 > 23.109781330317865.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.561077096649353 (T) = (0 -25.15340444021561) / Math.Sqrt((0.26644366816202597 / (20)) + (0.42317928135073474 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.15692575535330183 = (21.741589141590392 - 25.15340444021561) / 21.741589141590392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.11400783386274 > 6.562784703975959.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.413638829896183 (T) = (0 -8.017429105985608) / Math.Sqrt((0.012765924763554145 / (19)) + (0.04300602636820088 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.30004640193678783 = (6.16703303362201 - 8.017429105985608) / 6.16703303362201 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.18 ns 20.93 ns 1.22 0.03 False
5.33 ns 8.64 ns 1.62 0.44 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.931974140858518 > 18.159811563299478.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -61.12224373738485 (T) = (0 -20.80130125284261) / Math.Sqrt((0.04075506323424654 / (19)) + (0.008878317075821615 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.20725050380610646 = (17.23031068304566 - 20.80130125284261) / 17.23031068304566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.IsDefined_Generic_Flags

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.636123504825862 > 5.32821974887627.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -61.923117097116666 (T) = (0 -8.665733816155603) / Math.Sqrt((0.0596728455608157 / (20)) + (0.001568128898391612 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6798475753605462 = (5.158642928835775 - 8.665733816155603) / 5.158642928835775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.23 ns 21.14 ns 1.10 0.05 False
1.83 ns 4.05 ns 2.21 0.50 False
1.81 ns 4.06 ns 2.25 0.54 False
1.84 ns 4.06 ns 2.21 0.58 False
1.77 ns 4.09 ns 2.31 0.49 False
19.55 ns 21.54 ns 1.10 0.04 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

System.Tests.Perf_Char.Char_ToUpper(c: '你', cultureName: zh-Hans)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.135538030459998 > 20.3725300548747.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.901868321954515 (T) = (0 -21.309140548079686) / Math.Sqrt((0.0789506440730353 / (20)) + (0.0921445013528272 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.09330071586497374 = (19.49064903997688 - 21.309140548079686) / 19.49064903997688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToUpper(c: 'a', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.045885025090208 > 1.905075521626644.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/18/2023 7:10:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -53.056726400828715 (T) = (0 -4.08195480381395) / Math.Sqrt((0.031771046670970096 / (20)) + (0.0024124191915845133 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.3336691125474398 = (1.7491574884659105 - 4.08195480381395) / 1.7491574884659105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'a', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.058698442386175 > 1.8840523219217884.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -32.712473873369085 (T) = (0 -4.101012401010793) / Math.Sqrt((0.09356816068150547 / (20)) + (0.0027465437000227 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.3148918189098877 = (1.7715784243179074 - 4.101012401010793) / 1.7715784243179074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToUpper(c: 'A', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.064835874170707 > 1.722143122882878.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -62.30715103633122 (T) = (0 -4.092676177757789) / Math.Sqrt((0.024529727380501902 / (20)) + (0.001493021874901951 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.367802032840131 = (1.728470590444044 - 4.092676177757789) / 1.728470590444044 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'A', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.091207028805917 > 1.7332507491697637.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -57.42918700025848 (T) = (0 -4.1305618057144375) / Math.Sqrt((0.024637311419076872 / (20)) + (0.003616360310335926 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.3887619018252606 = (1.7291643016234737 - 4.1305618057144375) / 1.7291643016234737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToUpper(c: 'İ', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.544861903268355 > 20.93404921934878.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.371547004530946 (T) = (0 -21.425449191300935) / Math.Sqrt((0.028593852322716377 / (20)) + (0.06345398282437294 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.07932729716535207 = (19.85074337281268 - 21.425449191300935) / 19.85074337281268 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
32.45 ms 41.64 ms 1.28 0.11 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>.ConcurrentQueue(Size: 2000000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.64026842105263 > 34.120252211538464.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.757796929036912 (T) = (0 -42633320.02715121) / Math.Sqrt((895279267169.1927 / (20)) + (2708405396171.03 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.29438556239766545 = (32937110.29052197 - 42633320.02715121) / 32937110.29052197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.72 ns 8.05 ns 2.96 0.67 False
32.09 μs 39.96 μs 1.25 0.19 False
2.38 ns 8.07 ns 3.39 0.58 False
6.96 ns 10.11 ns 1.45 0.58 False
7.01 ns 10.01 ns 1.43 0.52 False
2.74 ns 8.03 ns 2.93 0.61 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

PerfLabTests.CastingPerf.CheckArrayIsArrayByVariance

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.052260136918294 > 2.8395470643773724.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.742828272190645 (T) = (0 -8.243706164773608) / Math.Sqrt((0.10637178448560418 / (20)) + (0.9763402605870227 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.1300149471964556 = (2.633759360209276 - 8.243706164773608) / 2.633759360209276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.95883354483926 > 35.53617241473429.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 6/1/2023 5:52:46 AM, 6/19/2023 11:51:49 PM, 7/1/2023 10:14:52 PM, 7/13/2023 7:10:54 AM, 7/27/2023 8:47:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -3.711650248728058 (T) = (0 -38553.00085299059) / Math.Sqrt((10400960.307107566 / (14)) + (8732391.774515854 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.1293147554857255 = (34138.401774807855 - 38553.00085299059) / 34138.401774807855 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsVariantGenericInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.067806321574569 > 2.6853723419399125.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -71.37674279082641 (T) = (0 -8.028335530519188) / Math.Sqrt((0.10244194689867746 / (20)) + (0.001508828363831968 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.8533941740103768 = (2.8136090006925176 - 8.028335530519188) / 2.8136090006925176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsNonvariantGenericInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.105750774357507 > 7.37539412882331.
IsChangePoint: Marked as a change because one of 7/29/2023 6:24:06 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.064419675353765 (T) = (0 -10.321954633377366) / Math.Sqrt((0.1894457448203466 / (19)) + (0.2527364395259588 / (8))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (8) - 2, .025) and -0.46475006143094705 = (7.046905069451657 - 10.321954633377366) / 7.046905069451657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsNonvariantGenericInterface

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.013771861675535 > 7.403907461629265.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.429893692704549 (T) = (0 -10.507307788260505) / Math.Sqrt((0.05146341304539987 / (17)) + (4.345864575305318 / (10))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (10) - 2, .025) and -0.5194281902897923 = (6.915303964616125 - 10.507307788260505) / 6.915303964616125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckListIsVariantGenericInterface

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.030268930294246 > 2.723737495523563.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/23/2023 10:00:36 AM, 4/24/2023 1:33:55 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -55.57245130200001 (T) = (0 -8.027569332323774) / Math.Sqrt((0.178582908420898 / (20)) + (0.002857539959931127 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.0206075294463033 = (2.6576009144078636 - 8.027569332323774) / 2.6576009144078636 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.42 ns 13.19 ns 1.27 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.194391969732173 > 10.9439471812029.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.276802761745254 (T) = (0 -12.994969909312536) / Math.Sqrt((1.1222345485350649 / (19)) + (0.07542103180828899 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.29757234375001573 = (10.014832677272356 - 12.994969909312536) / 10.014832677272356 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.52 ns 4.80 ns 3.16 0.59 False
1.57 ns 4.83 ns 3.07 0.61 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.IsEmpty<Int32>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.7986434522466 > 1.8549705456147634.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.51461031764043 (T) = (0 -4.657523737769106) / Math.Sqrt((0.15171513026341624 / (20)) + (0.09039288148748721 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.9530825575364763 = (1.577173562548319 - 4.657523737769106) / 1.577173562548319 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<Int32>.Stack(Size: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.828878563611595 > 1.6162054734224056.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -48.28476543073696 (T) = (0 -4.789039621204353) / Math.Sqrt((0.0872410369538361 / (19)) + (0.0024244503396260663 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.430467640546624 = (1.3960311313244886 - 4.789039621204353) / 1.3960311313244886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.57 ns 8.56 ns 1.54 0.46 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.557936868949326 > 5.850613411536316.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.313514631794927 (T) = (0 -8.452266282254964) / Math.Sqrt((0.26754675651038745 / (19)) + (0.040750426100961745 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5934658360586914 = (5.304328521508161 - 8.452266282254964) / 5.304328521508161 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.55 ns 4.83 ns 3.13 0.59 False
1.59 ns 4.82 ns 3.03 0.56 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.IsEmpty<String>.Stack(Size: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.831584710095078 > 1.6283317836584874.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -40.77740763797563 (T) = (0 -4.794028427474475) / Math.Sqrt((0.11553006986442767 / (20)) + (0.0023792482914839906 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.9870569186204161 = (1.60493373848685 - 4.794028427474475) / 1.60493373848685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<String>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.821710017581102 > 1.6431519087540654.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -32.0558853449128 (T) = (0 -4.807961332657084) / Math.Sqrt((0.21466955803116552 / (20)) + (0.00118635491353521 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -2.2913777542789275 = (1.4607746942466675 - 4.807961332657084) / 1.4607746942466675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
191.50 ns 210.69 ns 1.10 0.01 True
191.37 ns 212.03 ns 1.11 0.04 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Payloads

Baseline
Compare

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReference

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.68513205828785 > 202.49876880088303.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.64983650866315 (T) = (0 -210.16902085216475) / Math.Sqrt((0.9996521305685901 / (19)) + (13.027656938214983 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.09082582673300398 = (192.66964138685248 - 210.16902085216475) / 192.66964138685248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 212.02684685800676 > 201.07184017732547.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/11/2023 7:02:02 PM, 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.613391829161705 (T) = (0 -212.49711799016222) / Math.Sqrt((0.24063114479943434 / (20)) + (13.978415795558947 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.10799954490965792 = (191.78448129009695 - 212.49711799016222) / 191.78448129009695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Type

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.60 ns 1.79 ns 2.95 0.64 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Type*'

Payloads

Baseline
Compare

System.Tests.Perf_Type.op_Equality

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7874213512896842 > 0.848111653366943.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/8/2023 9:53:14 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.304077880541573 (T) = (0 -1.7483038815780412) / Math.Sqrt((0.12704502800019213 / (20)) + (0.04174518115409128 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.8644184881071286 = (0.9377207385199371 - 1.7483038815780412) / 0.9377207385199371 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.12 ns 20.37 ns 1.12 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.373502777237277 > 18.9910095072301.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -29.744709813346844 (T) = (0 -20.43925929248485) / Math.Sqrt((0.015117866572895454 / (20)) + (0.03934921235256466 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.13149613776347624 = (18.06392316361348 - 20.43925929248485) / 18.06392316361348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_SemaphoreSlim

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
43.10 ns 48.62 ns 1.13 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_SemaphoreSlim*'

Payloads

Baseline
Compare

System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWaitAsync

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.61979215872103 > 45.79517705316909.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.980738528549093 (T) = (0 -48.37731881021477) / Math.Sqrt((3.2474885253918067 / (19)) + (0.3873421035653526 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.09685415785812129 = (44.10551618337613 - 48.37731881021477) / 44.10551618337613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.13 ns 13.28 ns 1.19 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

System.Tests.Perf_String.IndexOfAny

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.278638297295261 > 11.688874082059812.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/11/2023 10:01:37 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -78.92191500118312 (T) = (0 -13.315193933892292) / Math.Sqrt((0.009101680916755513 / (20)) + (0.0023751383072333717 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.200563559204388 = (11.090786349301036 - 13.315193933892292) / 11.090786349301036 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.19 ns 7.78 ns 1.50 0.43 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_Path.HasExtension

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.781658852496203 > 5.445158671101189.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/22/2023 10:35:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.82232964188119 (T) = (0 -7.589912268507445) / Math.Sqrt((0.04283441581965235 / (20)) + (0.2630585236771347 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.5076133415366059 = (5.034389162921292 - 7.589912268507445) / 5.034389162921292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.17 ns 20.59 ns 1.13 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.590996984005876 > 18.99894406477629.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.717018291964663 (T) = (0 -20.55372354664296) / Math.Sqrt((0.014164959201200755 / (20)) + (0.07686445226169089 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.13571104356039007 = (18.09766988107133 - 20.55372354664296) / 18.09766988107133 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.39 ns 12.69 ns 1.22 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.686644130907272 > 10.944651230143785.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.960488439284907 (T) = (0 -12.913061789716682) / Math.Sqrt((0.004818660601718355 / (19)) + (0.367070496273132 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.2419863705856746 = (10.39710426421778 - 12.913061789716682) / 10.39710426421778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.22 ns 5.70 ns 1.77 0.49 False
0.24 ns 1.30 ns 5.33 0.67 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.702706710244053 > 3.487056597219071.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -44.461778945880305 (T) = (0 -5.632584832710931) / Math.Sqrt((0.013913859738327526 / (20)) + (0.013355718813154291 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6744319407633013 = (3.363878038628001 - 5.632584832710931) / 3.363878038628001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_String

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.29697336407987 > 0.25547725758885725.
IsChangePoint: Marked as a change because one of 5/3/2023 11:00:11 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.380204741310964 (T) = (0 -1.3680402135090077) / Math.Sqrt((0.09595348417757467 / (20)) + (0.0153812593933631 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -3.113068335339651 = (0.33260818979221674 - 1.3680402135090077) / 0.33260818979221674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.23 ns 20.62 ns 1.13 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.622818310436077 > 19.02789737083161.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.790271641020844 (T) = (0 -20.672134236255005) / Math.Sqrt((0.015504515972840053 / (20)) + (0.07082360227758981 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.14306685332095287 = (18.08479895659317 - 20.672134236255005) / 18.08479895659317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.01 ns 12.67 ns 1.15 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.669206200683176 > 11.754970869092185.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.77729048962417 (T) = (0 -12.579272063042263) / Math.Sqrt((0.187569801264726 / (20)) + (0.03779259773460203 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.14087014712481047 = (11.026033150875406 - 12.579272063042263) / 11.026033150875406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Aug 3, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
734.74 ms 1.91 secs 2.60 0.68 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>.ConcurrentStack(Size: 2000000)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.91167773 > 864.6680460526316.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.184622059517034 (T) = (0 -2182632371.649958) / Math.Sqrt((2.5273873130663456E+17 / (20)) + (93333935281626660 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -1.129797440978543 = (1024807490.9166667 - 2182632371.649958) / 1024807490.9166667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.22 ns 8.70 ns 1.40 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector4.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.70221555465846 > 6.536014229119455.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.307505861050586 (T) = (0 -8.583377433384426) / Math.Sqrt((0.010227900316287217 / (20)) + (0.08352877184021446 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.3585563001000837 = (6.318013786217101 - 8.583377433384426) / 6.318013786217101 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.24 ns 7.96 ns 1.52 0.18 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.Count<Int32>.Queue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.959425807710787 > 5.463750125008922.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -31.761002016874272 (T) = (0 -7.968916464335294) / Math.Sqrt((0.12315522137066404 / (20)) + (0.004245167413998091 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.48764350944781937 = (5.356737964254071 - 7.968916464335294) / 5.356737964254071 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.60 ns 13.24 ns 1.25 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.24396359758107 > 11.10857198831189.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.672203140459507 (T) = (0 -12.87124240213645) / Math.Sqrt((0.01191265751887867 / (20)) + (0.17374997724068386 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.22209730576612133 = (10.532092936795724 - 12.87124240213645) / 10.532092936795724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.39 ns 12.67 ns 1.22 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.671319889311453 > 10.917030888852759.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.061168038585183 (T) = (0 -13.042653722079512) / Math.Sqrt((0.008704927278788199 / (20)) + (0.11893535446733759 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.2390381506624455 = (10.526434327390422 - 13.042653722079512) / 10.526434327390422 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.92 ns 20.55 ns 1.15 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.55048526548116 > 19.020834003049494.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -46.17200795135756 (T) = (0 -20.597664395714304) / Math.Sqrt((0.01596651547417441 / (20)) + (0.014733312276101878 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.13737018198067558 = (18.109903637393113 - 20.597664395714304) / 18.109903637393113 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo mentioned this issue Aug 3, 2023
Closed
@EgorBo EgorBo closed this as completed Aug 3, 2023
@EgorBo EgorBo removed the untriaged label Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-arm64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EgorBo