Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 18 Improvements on 10/11/2022 6:23:40 AM #9015

Closed
performanceautofiler bot opened this issue Oct 18, 2022 · 1 comment
Closed

[Perf] Windows/x64: 18 Improvements on 10/11/2022 6:23:40 AM #9015

performanceautofiler bot opened this issue Oct 18, 2022 · 1 comment
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 28.21 ns 24.20 ns 0.86 0.03 False
TryFormat - Duration of single invocation 13.25 ns 10.72 ns 0.81 0.02 False
ToString - Duration of single invocation 17.21 ns 13.57 ns 0.79 0.02 False
TryFormat - Duration of single invocation 18.74 ns 15.41 ns 0.82 0.04 False
ToString - Duration of single invocation 11.92 ns 10.55 ns 0.89 0.03 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Int32.ToString(value: -2147483648)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.20152318399521 < 26.63268876516692.
IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 44.24026231354615 (T) = (0 -23.977790399080398) / Math.Sqrt((3.4066550689451636 / (299)) + (0.036829873079736244 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.1714686246075758 = (28.94011151686754 - 23.977790399080398) / 28.94011151686754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.718905305966235 < 12.433974778375598.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 84.41251242974879 (T) = (0 -10.107875298841027) / Math.Sqrt((1.0199997564251415 / (299)) + (0.05082712185715078 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.37126679144544195 = (16.076572958630226 - 10.107875298841027) / 16.076572958630226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.567968475969192 < 16.524403262619373.
IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 124.42512357736807 (T) = (0 -13.556252454735262) / Math.Sqrt((0.22881763913396713 / (299)) + (0.02505403837824304 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2653255245765609 = (18.452053131316344 - 13.556252454735262) / 18.452053131316344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.407233940188066 < 18.05969556543191.
IsChangePoint: Marked as a change because one of 8/16/2022 10:54:40 PM, 8/17/2022 2:11:12 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 58.93733117576123 (T) = (0 -15.023429373636016) / Math.Sqrt((0.7459431817667482 / (299)) + (0.06825605403783484 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.21065635166797841 = (19.03281214130542 - 15.023429373636016) / 19.03281214130542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.553691465721428 < 11.312543502248925.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/30/2022 9:31:11 PM, 10/4/2022 3:13:24 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.52056299021624 (T) = (0 -10.945843710244183) / Math.Sqrt((0.20086686106288093 / (299)) + (0.06912123152539784 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.12550641319878694 = (12.516779854592983 - 10.945843710244183) / 12.516779854592983 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormatL - Duration of single invocation 60.47 ns 38.54 ns 0.64 0.03 False
ToString4 - Duration of single invocation 38.37 ns 29.67 ns 0.77 0.15 False
TryFormat4 - Duration of single invocation 23.77 ns 20.07 ns 0.84 0.08 False
TryFormat3 - Duration of single invocation 19.50 ns 17.79 ns 0.91 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Version.TryFormatL

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.54064688166646 < 57.539743299622515.
IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 261.59784983390637 (T) = (0 -38.714039534153635) / Math.Sqrt((1.8930746424334433 / (299)) + (0.03636917818522335 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.36866103334227224 = (61.32052919068743 - 38.714039534153635) / 61.32052919068743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString4

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.66937186428705 < 36.05754443568543.
IsChangePoint: Marked as a change because one of 8/20/2022 4:00:21 AM, 9/16/2022 6:13:21 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 35.11392058340389 (T) = (0 -29.008670668646197) / Math.Sqrt((12.838101252873658 / (299)) + (0.2953841693668544 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.21659094548971863 = (37.02876613645967 - 29.008670668646197) / 37.02876613645967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat4

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.0668195860825 < 22.917815019980686.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/1/2022 8:04:48 PM, 9/23/2022 1:16:35 AM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 15.914623091143511 (T) = (0 -20.594347088086923) / Math.Sqrt((0.9783163687905226 / (299)) + (0.35133424225375265 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.08445993335352005 = (22.494206248691764 - 20.594347088086923) / 22.494206248691764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat3

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.78792832602109 < 18.513681691650394.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 24.188383752190646 (T) = (0 -17.546196258327857) / Math.Sqrt((0.5089705408614148 / (299)) + (0.025049075187107535 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.06430207958290811 = (18.751988088748295 - 17.546196258327857) / 18.751988088748295 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDecimal - Duration of single invocation 42.95 ns 34.45 ns 0.80 0.04 False
FormatterInt32 - Duration of single invocation 11.10 ns 9.52 ns 0.86 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.4462226280571 < 40.81218548359889.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 11:20:35 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 21.665335081352637 (T) = (0 -36.735600983957376) / Math.Sqrt((4.110046685325836 / (299)) + (0.7322433369982612 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.10142449981335945 = (40.882041605104 - 36.735600983957376) / 40.882041605104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.521979655478974 < 10.548402848186978.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/13/2022 12:23:36 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.896039067865765 (T) = (0 -10.03124920038653) / Math.Sqrt((0.13879326564997693 / (299)) + (0.05888005941145685 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.12515525330289862 = (11.466319296376437 - 10.03124920038653) / 11.466319296376437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 40.46 ns 28.86 ns 0.71 0.02 False
ToString - Duration of single invocation 32.98 ns 23.64 ns 0.72 0.02 False
TryFormat - Duration of single invocation 38.26 ns 21.78 ns 0.57 0.04 False
ToString - Duration of single invocation 15.46 ns 14.07 ns 0.91 0.06 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.856306561026752 < 38.78586561857362.
IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 95.61490412280253 (T) = (0 -29.532505395157497) / Math.Sqrt((1.0880888906099009 / (299)) + (0.422174522264563 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.29579598157329107 = (41.93742810661217 - 29.532505395157497) / 41.93742810661217 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.640936817121805 < 32.03760174921214.
IsChangePoint: Marked as a change because one of 9/15/2022 3:54:23 PM, 9/16/2022 2:24:22 AM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 29.99456321513672 (T) = (0 -23.87151819867576) / Math.Sqrt((33.901816220500585 / (299)) + (0.012523239306037682 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2976665088470188 = (33.988864975650294 - 23.87151819867576) / 33.988864975650294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.77573823006988 < 36.36757063873417.
IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 187.96752121486654 (T) = (0 -22.20986847936485) / Math.Sqrt((0.1455042220298712 / (136)) + (0.21188163592581852 / (31))) is greater than 1.974445630092179 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (136) + (31) - 2, .975) and 0.4293725386727912 = (38.92183602188974 - 22.20986847936485) / 38.92183602188974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.072477493062484 < 14.293331266209128.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 8/29/2022 8:26:03 AM, 10/1/2022 9:48:16 AM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 19.110555074674757 (T) = (0 -13.674684511074336) / Math.Sqrt((0.33930991196894184 / (299)) + (0.060370477085951724 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.07133744366394618 = (14.72513823000084 - 13.674684511074336) / 14.72513823000084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 16.51 ns 12.31 ns 0.75 0.02 False
TryFormat - Duration of single invocation 13.04 ns 8.37 ns 0.64 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_UInt32.ToString(value: 4294967295)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.31377077273517 < 15.637761136074626.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 111.75108432402597 (T) = (0 -12.581480542773399) / Math.Sqrt((0.5664865999881358 / (299)) + (0.03378338107588281 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.32544267434051805 = (18.651462320823654 - 12.581480542773399) / 18.651462320823654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.3678148052879 < 12.357675977252036.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 115.93937493021325 (T) = (0 -8.44402111093051) / Math.Sqrt((1.409018723815664 / (299)) + (0.002056057252829769 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.48695842747034934 = (16.458746353235338 - 8.44402111093051) / 16.458746353235338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline d6fdb3ab48ba885e5386a529ce53e143d56ab963
Compare d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3
Diff Diff

Improvements in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 22.85 ns 21.36 ns 0.93 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Int16.ToString(value: -32768)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.36172053982918 < 21.616268602878087.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 9/28/2022 11:19:50 AM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsImprovementStdDev: Marked as improvement because 27.37366719560002 (T) = (0 -20.9890588686797) / Math.Sqrt((1.5606145207149122 / (299)) + (0.042229008324285104 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.0954014540296165 = (23.20262282332574 - 20.9890588686797) / 23.20262282332574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
This was referenced Oct 18, 2022
Open
Closed
Open
Closed
Open
Open
Closed
Closed
Closed
@AndyAyersMS
Copy link
Member

dotnet/runtime#76726

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@AndyAyersMS