Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 146 Improvements on 11/2/2022 4:25:40 PM #9800

Closed
performanceautofiler bot opened this issue Nov 15, 2022 · 7 comments
Closed

[Perf] Linux/x64: 146 Improvements on 11/2/2022 4:25:40 PM #9800

performanceautofiler bot opened this issue Nov 15, 2022 · 7 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-jit os-linux perf-improvement PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAllBenchmark - Duration of single invocation 18.23 ns 2.40 ns 0.13 0.02 False
LessThanAnyBenchmark - Duration of single invocation 18.92 ns 2.17 ns 0.11 0.15 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 19.73 ns 3.02 ns 0.15 0.18 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 20.37 ns 15.06 ns 0.74 0.16 False
GreaterThanAnyBenchmark - Duration of single invocation 18.86 ns 11.49 ns 0.61 0.03 False
LessThanOrEqualAllBenchmark - Duration of single invocation 19.73 ns 12.69 ns 0.64 0.06 False
LessThanAllBenchmark - Duration of single invocation 19.16 ns 11.59 ns 0.61 0.16 False
EqualsAnyBenchmark - Duration of single invocation 18.85 ns 11.30 ns 0.60 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 20.38 ns 2.60 ns 0.13 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.395168157075718 < 17.317562693670112.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 560.5575442345457 (T) = (0 -2.329593210496832) / Math.Sqrt((0.108593002524139 / (299)) + (0.011914096100165828 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.8710944107089001 = (18.0720884432408 - 2.329593210496832) / 18.0720884432408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1717122926272756 < 17.957612159453145.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 329.9617334898076 (T) = (0 -1.9054865108382946) / Math.Sqrt((0.41464278790132053 / (299)) + (0.03320867904781596 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.8977854452319234 = (18.642027206025755 - 1.9054865108382946) / 18.642027206025755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.0172193211715417 < 18.739294246005947.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 218.59713604594896 (T) = (0 -2.79392735997018) / Math.Sqrt((0.41645998424845154 / (299)) + (0.12542835687828185 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.8570545610811261 = (19.545411040053008 - 2.79392735997018) / 19.545411040053008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.058136810575428 < 19.37810314164331.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 30.319851890336334 (T) = (0 -15.475602740956433) / Math.Sqrt((0.3730398700070579 / (299)) + (0.6321598714236657 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.23219166335845337 = (20.155554456009064 - 15.475602740956433) / 20.155554456009064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.49120141048872 < 17.9269581406431.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 119.0766634458217 (T) = (0 -11.267130401508194) / Math.Sqrt((0.08930478681587584 / (299)) + (0.10208723591607755 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.39895538730684865 = (18.745913636963838 - 11.267130401508194) / 18.745913636963838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.686959730173337 < 18.749126218101598.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 12.865792976763379 (T) = (0 -15.184413073842991) / Math.Sqrt((0.7341844093637053 / (299)) + (2.708691454758906 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.21064631641401046 = (19.236513858858622 - 15.184413073842991) / 19.236513858858622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.58954701025852 < 17.329446582752084.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 30.34804285194759 (T) = (0 -12.6390203033228) / Math.Sqrt((0.412836383563778 / (299)) + (0.9037002953976031 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3057970379397734 = (18.206520274435654 - 12.6390203033228) / 18.206520274435654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.302627784078792 < 17.924418364700852.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 128.32299104517412 (T) = (0 -11.216345487581444) / Math.Sqrt((0.1053409203223075 / (299)) + (0.08591870014769262 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.40089030644689133 = (18.721689213641742 - 11.216345487581444) / 18.721689213641742 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5973918293352383 < 19.354126004863296.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 276.3201866241826 (T) = (0 -2.762291407850752) / Math.Sqrt((0.7406088971622274 / (299)) + (0.03820454197492705 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.8611100758036608 = (19.888349884516394 - 2.762291407850752) / 19.888349884516394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAnyBenchmark - Duration of single invocation 32.88 ns 18.64 ns 0.57 0.29 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 33.37 ns 26.13 ns 0.78 0.34 False
GreaterThanAllBenchmark - Duration of single invocation 26.13 ns 2.24 ns 0.09 0.30 False
DivideBenchmark - Duration of single invocation 51.11 ns 46.12 ns 0.90 0.28 False
MultiplyOperatorBenchmark - Duration of single invocation 26.62 ns 24.38 ns 0.92 0.50 False
EqualsAnyBenchmark - Duration of single invocation 32.91 ns 18.11 ns 0.55 0.31 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 33.62 ns 2.70 ns 0.08 0.45 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 33.40 ns 2.72 ns 0.08 0.35 False
MultiplyBenchmark - Duration of single invocation 33.47 ns 24.29 ns 0.73 0.47 False
LessThanAllBenchmark - Duration of single invocation 26.32 ns 18.31 ns 0.70 0.29 False
LessThanAnyBenchmark - Duration of single invocation 32.83 ns 2.25 ns 0.07 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.635095451488482 < 25.477055597267015.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 45.616989031185675 (T) = (0 -19.32584662907559) / Math.Sqrt((4.694649436813596 / (299)) + (0.39300379318149364 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.28928801320981695 = (27.192233968583647 - 19.32584662907559) / 27.192233968583647 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.127623351017487 < 31.71569018608524.
IsChangePoint: Marked as a change because one of 7/21/2022 3:32:26 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 19.29731603117264 (T) = (0 -25.838257293045405) / Math.Sqrt((3.9296696628139505 / (299)) + (2.681046669902644 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1977243017837162 = (32.20620710622563 - 25.838257293045405) / 32.20620710622563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.23564564763084 < 24.838370835270442.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 167.70665560708753 (T) = (0 -2.0961341430829243) / Math.Sqrt((5.9779187436439765 / (299)) + (0.038343467453535465 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9217222806143865 = (26.77817084522993 - 2.0961341430829243) / 26.77817084522993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.116218733197066 < 48.21761288133621.
IsChangePoint: Marked as a change because one of 9/16/2022 6:13:21 PM, 9/26/2022 12:57:03 PM, 11/1/2022 10:53:48 PM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 10.768230936734298 (T) = (0 -45.013570760745225) / Math.Sqrt((10.24701231324035 / (299)) + (2.8203430784899086 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08289029210001145 = (49.081991361554735 - 45.013570760745225) / 49.081991361554735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.379920125260153 < 26.967562364985795.
IsChangePoint: Marked as a change because one of 7/28/2022 3:42:13 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 32.033413207676176 (T) = (0 -24.360067902381765) / Math.Sqrt((8.681732281183 / (299)) + (0.007301877486907519 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18372573532256195 = (29.842993901085414 - 24.360067902381765) / 29.842993901085414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.105636045101175 < 31.2616449646187.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 40.49101304850573 (T) = (0 -19.090174200983455) / Math.Sqrt((6.530778887710052 / (299)) + (0.4896359935445205 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.3057944205670934 = (27.499309666422057 - 19.090174200983455) / 27.499309666422057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6974213698042533 < 31.910497314584823.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 239.102931950074 (T) = (0 -2.3919718403100094) / Math.Sqrt((4.265196910710571 / (299)) + (0.0306471019112691 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.9253086214483268 = (32.02473815174251 - 2.3919718403100094) / 32.02473815174251 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.715517386358525 < 29.7809741548428.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 248.81212071026079 (T) = (0 -2.298764258140314) / Math.Sqrt((3.8037632831261337 / (299)) + (0.04295905850090539 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.9281780245942727 = (32.006419277030936 - 2.298764258140314) / 32.006419277030936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.28886777040557 < 25.194890799971773.
IsChangePoint: Marked as a change because one of 7/28/2022 9:13:34 AM, 11/2/2022 4:14:20 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 33.42746809782908 (T) = (0 -24.33586455207027) / Math.Sqrt((8.13468856687658 / (299)) + (0.0014726277675215908 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.18485505784395434 = (29.85464706154256 - 24.33586455207027) / 29.85464706154256 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.31242882975064 < 24.9016943297956.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 38.48820549931714 (T) = (0 -19.048080703733966) / Math.Sqrt((8.19395446141325 / (299)) + (0.3459881878613525 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.28719515219446035 = (26.72271486701571 - 19.048080703733966) / 26.72271486701571 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2535619895954744 < 29.61332068679596.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 143.29546952005782 (T) = (0 -2.353531832062184) / Math.Sqrt((8.338358236546023 / (299)) + (0.07368018789443707 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.9140631862808282 = (27.386770933271524 - 2.353531832062184) / 27.386770933271524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AbsBenchmark - Duration of single invocation 20.93 ns 19.12 ns 0.91 0.02 False
GreaterThanAnyBenchmark - Duration of single invocation 15.60 ns 7.44 ns 0.48 0.03 False
EqualsAnyBenchmark - Duration of single invocation 15.60 ns 7.44 ns 0.48 0.02 False
GreaterThanAllBenchmark - Duration of single invocation 14.92 ns 3.20 ns 0.21 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Single>.AbsBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.123040096570115 < 19.887654215375402.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 110.33239804733492 (T) = (0 -19.16486418475705) / Math.Sqrt((0.029798834141876927 / (299)) + (0.003907848412778065 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08635891652508329 = (20.976359898206358 - 19.16486418475705) / 20.976359898206358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.444105043408356 < 14.867554875411667.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 32.51688823965737 (T) = (0 -8.684460947998957) / Math.Sqrt((0.04176563404174014 / (299)) + (1.2223841680610237 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.43933061923461436 = (15.48945108460097 - 8.684460947998957) / 15.48945108460097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.441737306595874 < 14.839632276101304.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 35.4522411485712 (T) = (0 -9.07364489923945) / Math.Sqrt((0.032710367399350175 / (299)) + (0.924135807009926 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.4155463372192714 = (15.52500305339628 - 9.07364489923945) / 15.52500305339628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1955114712997177 < 14.14414275327543.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 247.28092341038507 (T) = (0 -2.997799582449405) / Math.Sqrt((0.049461252253359574 / (299)) + (0.056631465503534734 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.7969094117718212 = (14.760898614766337 - 2.997799582449405) / 14.760898614766337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 221.19 μs 205.07 μs 0.93 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 205.07142988888884 < 211.47098435363083.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 67.23402682929068 (T) = (0 -204685.28164706234) / Math.Sqrt((9270183.43244476 / (299)) + (849994.5819415507 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07524458923320096 = (221339.91243948397 - 204685.28164706234) / 221339.91243948397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 160.07 μs 147.88 μs 0.92 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;IndexViewModel&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 147.87775047281323 < 152.71525225652977.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 16.154254224648795 (T) = (0 -151648.96438552553) / Math.Sqrt((21559339.642280407 / (299)) + (11130811.811032591 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06803417579942585 = (162719.44791066524 - 151648.96438552553) / 162719.44791066524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 11.33 μs 9.60 μs 0.85 0.01 False
SerializeToString - Duration of single invocation 10.92 μs 9.20 μs 0.84 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.599809417582419 < 10.7409723998443.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 48.46879090380063 (T) = (0 -9588.833694540099) / Math.Sqrt((100246.08206380982 / (299)) + (20104.56985989449 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14092882820894428 = (11161.861798421873 - 9588.833694540099) / 11161.861798421873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.200843476846394 < 10.36748669095348.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 66.8085618484383 (T) = (0 -9156.877771902866) / Math.Sqrt((90205.4487205626 / (299)) + (6417.114752343346 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.14952694998978863 = (10766.80533474038 - 9156.877771902866) / 10766.80533474038 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 44.46 μs 40.44 μs 0.91 0.00 False
SerializeToString - Duration of single invocation 44.41 μs 40.32 μs 0.91 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.43974427921884 < 42.35041143353174.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 44.39213668277891 (T) = (0 -40329.95094932094) / Math.Sqrt((831044.1395631724 / (299)) + (38533.52137642429 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06843888882853358 = (43292.86663609734 - 40329.95094932094) / 43292.86663609734 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.32030442286451 < 41.80432483277832.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 35.54403127695132 (T) = (0 -39885.074655724384) / Math.Sqrt((848389.7171315993 / (299)) + (85908.81279894518 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06737974649391858 = (42766.682908483825 - 39885.074655724384) / 42766.682908483825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 82.33 μs 74.15 μs 0.90 0.01 False
SerializeToString - Duration of single invocation 81.59 μs 73.49 μs 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 74.1515631121643 < 78.26798371841018.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 42.37558003587104 (T) = (0 -73403.77241233461) / Math.Sqrt((3500108.7815541187 / (299)) + (255679.89197883444 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07692325005477653 = (79520.76836155876 - 73403.77241233461) / 79520.76836155876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.48958752031417 < 77.60084977557865.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 39.04245129360017 (T) = (0 -72968.05711053842) / Math.Sqrt((3620905.3340109433 / (299)) + (311085.90917498997 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07538722304265327 = (78917.4224378088 - 72968.05711053842) / 78917.4224378088 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in Burgers

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test3 - Duration of single invocation 3.38 secs 2.77 secs 0.82 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Burgers*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Burgers.Test3

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.770662704 < 3.2116707862886216.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 310.76042695209696 (T) = (0 -2780247513.400576) / Math.Sqrt((22600632778882.164 / (299)) + (102359792657959.19 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.17757045356831394 = (3380529706.7248707 - 2780247513.400576) / 3380529706.7248707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 5.49 μs 3.72 μs 0.68 0.11 False
MultipleSerial - Duration of single invocation 44.88 μs 30.54 μs 0.68 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.715738219166666 < 5.212173081678572.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 18.316485182284293 (T) = (0 -3715.5764428691796) / Math.Sqrt((780908.6705529388 / (299)) + (4.795018494424387 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.201238780065904 = (4651.673554176483 - 3715.5764428691796) / 4651.673554176483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;.MultipleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.543432772142854 < 42.577594744366756.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 16.890064014177195 (T) = (0 -31087.92169074633) / Math.Sqrt((36089548.22132407 / (299)) + (7251035.718505391 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.25080472468210196 = (41495.085079861354 - 31087.92169074633) / 41495.085079861354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiString - Duration of single invocation 149.74 ns 138.86 ns 0.93 0.01 False
WriteAsciiString - Duration of single invocation 12.40 ms 9.09 ms 0.73 0.01 False
WriteAsciiString - Duration of single invocation 30.08 μs 24.32 μs 0.81 0.01 False
WriteAsciiCharArray - Duration of single invocation 30.10 μs 24.32 μs 0.81 0.01 False
WriteAsciiCharArray - Duration of single invocation 7.59 ms 6.14 ms 0.81 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 32)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 138.8618400280007 < 142.16118879323776.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 102.40414436565587 (T) = (0 -138.99001454238078) / Math.Sqrt((2.6868614592397737 / (299)) + (0.024137869149610663 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06813436618848336 = (149.15242015512885 - 138.99001454238078) / 149.15242015512885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.087030464583334 < 11.798575858112978.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 27.452182109314357 (T) = (0 -9710388.568099778) / Math.Sqrt((61304351666.03463 / (299)) + (244971459143.17786 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.21105506912501798 = (12308068.900741195 - 9710388.568099778) / 12308068.900741195 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.321795924053912 < 28.585053136758457.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 162.35744210267893 (T) = (0 -24322.608853338465) / Math.Sqrt((338102.7784153541 / (299)) + (469.522839446415 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18442203381410027 = (29822.542861333826 - 24322.608853338465) / 29822.542861333826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.32497709953344 < 28.59978665205868.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 163.09636677402813 (T) = (0 -24316.460230931734) / Math.Sqrt((334131.17970084987 / (299)) + (601.4771214481829 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18457934485818617 = (29820.755799597373 - 24316.460230931734) / 29820.755799597373 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.135891199652779 < 7.2077593785515885.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 164.26474166132328 (T) = (0 -6134475.118756269) / Math.Sqrt((20379525725.533768 / (299)) + (72185191.6257566 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18381427403549006 = (7516028.427851988 - 6134475.118756269) / 7516028.427851988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyOperatorBenchmark - Duration of single invocation 14.15 ns 11.71 ns 0.83 0.00 False
AbsBenchmark - Duration of single invocation 12.79 ns 11.01 ns 0.86 0.00 False
LessThanOrEqualAllBenchmark - Duration of single invocation 11.74 ns 5.96 ns 0.51 0.00 False
LessThanAllBenchmark - Duration of single invocation 11.29 ns 5.35 ns 0.47 0.03 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 12.80 ns 6.07 ns 0.47 0.01 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 12.17 ns 2.13 ns 0.17 0.03 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.40 ns 2.24 ns 0.18 0.05 False
MultiplyBenchmark - Duration of single invocation 14.16 ns 11.68 ns 0.83 0.01 False
LessThanAnyBenchmark - Duration of single invocation 11.97 ns 2.05 ns 0.17 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int32>.MultiplyOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.709568684999544 < 13.423953872432456.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 425.12571735256256 (T) = (0 -11.705540536551018) / Math.Sqrt((0.003052982953455526 / (299)) + (0.0006314569782949769 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1721041431580246 = (14.138904597496147 - 11.705540536551018) / 14.138904597496147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.010551611434781 < 12.142942112299739.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 482.20198927142087 (T) = (0 -11.010228244913604) / Math.Sqrt((0.0007769541001279637 / (299)) + (0.00030127816108222746 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.13798323126262732 = (12.772638125173232 - 11.010228244913604) / 12.772638125173232 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.959901787950923 < 11.158320308436581.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 484.04991141831357 (T) = (0 -6.027577493154518) / Math.Sqrt((0.00035241837946868614 / (299)) + (0.00316864598207451 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4862858871896138 = (11.733330548735225 - 6.027577493154518) / 11.733330548735225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.353883497387135 < 10.726852530760247.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 151.06589910896696 (T) = (0 -5.56517364330131) / Math.Sqrt((0.03136161108568679 / (299)) + (0.029756161133098157 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5037645596745929 = (11.214784739380844 - 5.56517364330131) / 11.214784739380844 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.067064131431481 < 12.174734275019267.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 287.2623690990795 (T) = (0 -5.991648293323228) / Math.Sqrt((0.06486929797783926 / (299)) + (0.00756296628929043 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5283149302744324 = (12.702645637711717 - 5.991648293323228) / 12.702645637711717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1278947200798606 < 11.572455598897774.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 247.11037364388181 (T) = (0 -1.950612799011959) / Math.Sqrt((0.0632530367753014 / (299)) + (0.033526155817144215 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8380769061400634 = (12.046538591334214 - 1.950612799011959) / 12.046538591334214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2353351575418667 < 11.773462806913734.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 154.93897610367392 (T) = (0 -2.485938638615098) / Math.Sqrt((0.0007626541728745831 / (299)) + (0.11450121542560605 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.799473282963565 = (12.397044520323998 - 2.485938638615098) / 12.397044520323998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.684355757129865 < 13.450724786342889.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 293.58324199490926 (T) = (0 -11.701421462109485) / Math.Sqrt((0.0164014480471696 / (299)) + (0.0004192610017042852 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.17331869085613855 = (14.154694599576546 - 11.701421462109485) / 14.154694599576546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.048583454417334 < 11.36193905537297.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 382.3869096801894 (T) = (0 -2.043277378358042) / Math.Sqrt((0.02811242331680177 / (299)) + (0.013066656545445615 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8280484367248474 = (11.88286596201769 - 2.043277378358042) / 11.88286596201769 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 1.07 ms 803.24 μs 0.75 0.01 False
GetByteCount - Duration of single invocation 396.45 μs 243.88 μs 0.62 0.01 False
GetString - Duration of single invocation 1.13 ms 905.07 μs 0.80 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 803.2433402083333 < 1.0137179968343255.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 41.99867192296771 (T) = (0 -838536.2781132469) / Math.Sqrt((462030915.5439399 / (299)) + (726860206.9236007 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.20802951094595956 = (1058797.3790725793 - 838536.2781132469) / 1058797.3790725793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 243.88458550824174 < 376.7241162641655.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 20.558214686477253 (T) = (0 -280780.9920527478) / Math.Sqrt((77419365.25034836 / (299)) + (672731213.0460982 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.2845542628790662 = (392456.02773825265 - 280780.9920527478) / 392456.02773825265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 905.0679495370372 < 1.0702857131797405.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 55.74279774515554 (T) = (0 -901459.4064032452) / Math.Sqrt((1430046112.0217016 / (299)) + (265851086.96612164 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18939803942459077 = (1112086.387952159 - 901459.4064032452) / 1112086.387952159 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanAnyBenchmark - Duration of single invocation 8.93 ns 4.19 ns 0.47 0.03 False
AbsBenchmark - Duration of single invocation 10.62 ns 8.88 ns 0.84 0.00 False
EqualsAnyBenchmark - Duration of single invocation 8.90 ns 4.19 ns 0.47 0.02 False
LessThanAllBenchmark - Duration of single invocation 8.30 ns 4.15 ns 0.50 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 8.28 ns 2.39 ns 0.29 0.03 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 8.70 ns 1.91 ns 0.22 0.06 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 9.32 ns 3.52 ns 0.38 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.186956584410348 < 8.45914232232762.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 201.72862826571665 (T) = (0 -4.213960673987489) / Math.Sqrt((0.04102212584123662 / (299)) + (0.011219673111639142 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5261266256418266 = (8.89258798238029 - 4.213960673987489) / 8.89258798238029 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.883824336719515 < 10.152913473349834.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 59.76147779789947 (T) = (0 -8.98800764394412) / Math.Sqrt((0.002506089029401812 / (299)) + (0.01812067730850422 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15797981578109402 = (10.674337518739865 - 8.98800764394412) / 10.674337518739865 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.194775072221944 < 8.471226179131692.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 278.8779574240584 (T) = (0 -4.186423119752446) / Math.Sqrt((0.008402784647643269 / (299)) + (0.007137413326833601 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5284478480096088 = (8.877964191408784 - 4.186423119752446) / 8.877964191408784 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.149910498842764 < 7.898886482957342.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 151.3630546742874 (T) = (0 -4.210294599626608) / Math.Sqrt((0.013233244365700808 / (299)) + (0.018627212035934308 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.4891729052901866 = (8.242112924762456 - 4.210294599626608) / 8.242112924762456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3932984585821737 < 7.8975929575925585.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 205.2120328786779 (T) = (0 -2.454245323577664) / Math.Sqrt((0.00615912623021518 / (299)) + (0.02191746095682108 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.703260782224934 = (8.270714407011848 - 2.454245323577664) / 8.270714407011848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9108847125917994 < 8.254634136810013.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 200.83358204000257 (T) = (0 -2.057876539782329) / Math.Sqrt((0.0632305015450598 / (299)) + (0.023576428712644832 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.7600544181421507 = (8.57643022158864 - 2.057876539782329) / 8.57643022158864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.5159493372542157 < 8.856879931221446.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 231.46007503898886 (T) = (0 -3.37079722911766) / Math.Sqrt((0.036223338641742556 / (299)) + (0.013974286539269205 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.634421280476197 = (9.22044158781563 - 3.37079722911766) / 9.22044158781563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAnyBenchmark - Duration of single invocation 30.40 ns 2.17 ns 0.07 0.14 False
GreaterThanAnyBenchmark - Duration of single invocation 28.25 ns 19.06 ns 0.67 0.28 False
LessThanAnyBenchmark - Duration of single invocation 32.42 ns 2.06 ns 0.06 0.29 False
LessThanOrEqualAllBenchmark - Duration of single invocation 29.69 ns 21.32 ns 0.72 0.19 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 36.69 ns 21.27 ns 0.58 0.38 False
GreaterThanAllBenchmark - Duration of single invocation 26.18 ns 1.78 ns 0.07 0.32 False
AbsBenchmark - Duration of single invocation 26.16 ns 21.09 ns 0.81 0.31 False
MultiplyBenchmark - Duration of single invocation 26.56 ns 24.32 ns 0.92 0.28 False
LessThanAllBenchmark - Duration of single invocation 26.20 ns 21.58 ns 0.82 0.28 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 29.59 ns 2.18 ns 0.07 0.47 False
EqualsAnyBenchmark - Duration of single invocation 26.91 ns 19.48 ns 0.72 0.30 False
MultiplyOperatorBenchmark - Duration of single invocation 26.57 ns 24.46 ns 0.92 0.30 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.167080067680549 < 28.754206545300377.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 377.7139821964916 (T) = (0 -2.2984080571189662) / Math.Sqrt((1.4749625911193254 / (299)) + (0.017822163346185662 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.9246098138491554 = (30.486833558418216 - 2.2984080571189662) / 30.486833558418216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.05655041215409 < 26.140349251966803.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 53.540027082830356 (T) = (0 -18.896418964571932) / Math.Sqrt((5.63787033900065 / (299)) + (0.1322271320670971 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.30768797000422077 = (27.294656377251073 - 18.896418964571932) / 27.294656377251073 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0638756860083243 < 25.535713055771954.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 168.77481805170999 (T) = (0 -2.15758769033966) / Math.Sqrt((6.313682814153734 / (299)) + (0.014166341469916468 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.9200150221221532 = (26.97491138441938 - 2.15758769033966) / 26.97491138441938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.32262316291508 < 28.19186734031299.
IsChangePoint: Marked as a change because one of 8/5/2022 4:49:35 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 21.28459954654275 (T) = (0 -23.507601004410752) / Math.Sqrt((1.8500777471771912 / (299)) + (2.4415896160020676 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.21672736772690976 = (30.012029063482917 - 23.507601004410752) / 30.012029063482917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.267791281540575 < 34.84193679868678.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 22.673551045314504 (T) = (0 -24.104805968730876) / Math.Sqrt((9.282656716010376 / (299)) + (3.9768737982772793 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.29814551346965856 = (34.34444949963117 - 24.104805968730876) / 34.34444949963117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.782293870351701 < 24.851748538987888.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 147.78266694155164 (T) = (0 -2.0195653244846157) / Math.Sqrt((7.913810858439054 / (299)) + (0.04442971135120502 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.924990272196681 = (26.924045502205598 - 2.0195653244846157) / 26.924045502205598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.090278387964442 < 24.833476348252905.
IsChangePoint: Marked as a change because one of 8/4/2022 9:08:56 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 25.26575826055385 (T) = (0 -21.150592925051964) / Math.Sqrt((2.9268050904018086 / (299)) + (0.4062587614523179 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.15697622140454984 = (25.08896363551057 - 21.150592925051964) / 25.08896363551057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.318314813130602 < 31.414632318499486.
IsChangePoint: Marked as a change because one of 9/14/2022 12:15:21 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 25.812287571458906 (T) = (0 -24.36766162855408) / Math.Sqrt((13.65498632011139 / (299)) + (0.00443007000571096 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18484713780777454 = (29.893364494876565 - 24.36766162855408) / 29.893364494876565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.578728291843785 < 24.92305377028241.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 25.183067061589313 (T) = (0 -20.00754525239504) / Math.Sqrt((7.117007110338729 / (299)) + (1.153337886920483 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.24293174594086994 = (26.427663747781942 - 20.00754525239504) / 26.427663747781942 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.175586827912549 < 28.142179155261946.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 151.1684999867284 (T) = (0 -2.1434144594307236) / Math.Sqrt((12.49916186399267 / (299)) + (0.040615147232970876 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.9363917877511213 = (33.69713412230837 - 2.1434144594307236) / 33.69713412230837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.47859304857859 < 25.51927284223048.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 51.16701586124303 (T) = (0 -19.105092054579185) / Math.Sqrt((6.102833642761798 / (299)) + (0.1259450750975956 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.30114045274664 = (27.33752744691767 - 19.105092054579185) / 27.33752744691767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.461337781618557 < 31.469654289209053.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 25.979601387584513 (T) = (0 -24.395882776865033) / Math.Sqrt((13.52063976649371 / (299)) + (0.008339029135412936 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.18513554910598798 = (29.938577821255528 - 24.395882776865033) / 29.938577821255528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAllBenchmark - Duration of single invocation 9.27 ns 3.76 ns 0.41 0.02 False
LessThanAllBenchmark - Duration of single invocation 9.22 ns 3.27 ns 0.35 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 9.93 ns 2.23 ns 0.22 0.04 False
LessThanAnyBenchmark - Duration of single invocation 9.87 ns 1.94 ns 0.20 0.04 False
AbsBenchmark - Duration of single invocation 10.82 ns 9.12 ns 0.84 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7622107056257996 < 8.791997985272733.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 62.65276776079071 (T) = (0 -4.239221105585136) / Math.Sqrt((0.026420917703319346 / (299)) + (0.14567085928615778 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.5422008899297216 = (9.260002940884611 - 4.239221105585136) / 9.260002940884611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.271978827128803 < 8.733738495343006.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 254.3308121089713 (T) = (0 -3.3353389198327084) / Math.Sqrt((0.026472608139715522 / (299)) + (0.00986436298662189 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.6343063612736108 = (9.120582276051561 - 3.3353389198327084) / 9.120582276051561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.227376827765515 < 9.435848174996678.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 177.58722606674132 (T) = (0 -2.463391023878692) / Math.Sqrt((0.02680587095635243 / (299)) + (0.03869980910484944 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.7521609829605046 = (9.939480285649005 - 2.463391023878692) / 9.939480285649005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.938862210159947 < 9.342557916775267.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 237.72916635461118 (T) = (0 -2.0720301071490423) / Math.Sqrt((0.04572254308226407 / (299)) + (0.020546446205188043 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.7877352768280318 = (9.761537744877035 - 2.0720301071490423) / 9.761537744877035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.118222483735709 < 10.277890369593356.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 113.76417278632336 (T) = (0 -8.959593280645429) / Math.Sqrt((0.00033193773363477373 / (299)) + (0.0075168522242010085 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1725102648325042 = (10.827437368551621 - 8.959593280645429) / 10.827437368551621 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadToEndAsync - Duration of single invocation 338.95 ms 295.47 ms 0.87 0.01 False
ReadToEndAsync - Duration of single invocation 338.30 ms 295.56 ms 0.87 0.01 False
ReadToEndAsync - Duration of single invocation 338.76 ms 296.10 ms 0.87 0.01 False
ReadToEndAsync - Duration of single invocation 339.19 ms 295.97 ms 0.87 0.01 False
ReadToEnd - Duration of single invocation 333.33 ms 289.37 ms 0.87 0.01 False
ReadToEnd - Duration of single invocation 333.47 ms 290.39 ms 0.87 0.01 False
ReadToEnd - Duration of single invocation 335.09 ms 290.39 ms 0.87 0.01 False
ReadToEndAsync - Duration of single invocation 338.77 ms 295.62 ms 0.87 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [ 129, 1024])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.4652174 < 321.98843097499997.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 66.44308736312983 (T) = (0 -295689511.8027074) / Math.Sqrt((79279751807442.28 / (299)) + (2446407083863.4043 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.12046363798383782 = (336187933.29351157 - 295689511.8027074) / 336187933.29351157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.5575053333333 < 322.19010804136906.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 71.50253318239184 (T) = (0 -295636107.3428093) / Math.Sqrt((82370450961866.8 / (299)) + (986638481166.0359 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.12008290307639494 = (335981774.1653412 - 295636107.3428093) / 335981774.1653412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 296.1029342 < 321.99341261666666.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 67.5109635083893 (T) = (0 -295982477.7056379) / Math.Sqrt((80016439734215.12 / (299)) + (1938902727036.2463 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11918244970119402 = (336031539.7953068 - 295982477.7056379) / 336031539.7953068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.97312276923077 < 321.9043681033333.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 70.65312212141585 (T) = (0 -295289230.9011944) / Math.Sqrt((80222130046629.19 / (299)) + (1470810122159.4749 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1212008532820593 = (336014471.5706812 - 295289230.9011944) / 336014471.5706812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 289.37189873333335 < 316.5560350183333.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 63.95195558376353 (T) = (0 -289971997.3163139) / Math.Sqrt((99100011388222.22 / (299)) + (1065906428455.1368 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1184133491121646 = (328920585.42888165 - 289971997.3163139) / 328920585.42888165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   1,    1])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 290.387811 < 316.52731769243593.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 65.54270063392892 (T) = (0 -289864526.76876634) / Math.Sqrt((101774683317980 / (299)) + (466144616201.1039 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.11902692456232863 = (329027679.5630336 - 289864526.76876634) / 329027679.5630336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 290.3907578571429 < 316.7353278714285.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 43.37556550200227 (T) = (0 -291047254.2658556) / Math.Sqrt((104047364880521.19 / (299)) + (11726190822250.123 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1154369829810768 = (329029417.5385238 - 291047254.2658556) / 329029417.5385238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEndAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.61999121428573 < 322.06776591833335.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 63.392202054396364 (T) = (0 -296193564.1120003) / Math.Sqrt((78317804073791.66 / (299)) + (3155537339561.926 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11910775687107023 = (336242674.8814595 - 296193564.1120003) / 336242674.8814595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 20.38 ns 12.59 ns 0.62 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 20.86 ns 2.21 ns 0.11 0.47 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 19.72 ns 1.97 ns 0.10 0.04 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 20.37 ns 2.26 ns 0.11 0.12 False
LessThanAllBenchmark - Duration of single invocation 18.24 ns 11.43 ns 0.63 0.05 False
LessThanAnyBenchmark - Duration of single invocation 18.86 ns 2.18 ns 0.12 0.03 False
EqualsAnyBenchmark - Duration of single invocation 19.31 ns 11.31 ns 0.59 0.38 False
MultiplyOperatorBenchmark - Duration of single invocation 20.46 ns 18.01 ns 0.88 0.01 False
MultiplyBenchmark - Duration of single invocation 20.48 ns 18.03 ns 0.88 0.02 False
GreaterThanAnyBenchmark - Duration of single invocation 18.88 ns 11.30 ns 0.60 0.38 False
LessThanOrEqualAllBenchmark - Duration of single invocation 19.73 ns 12.66 ns 0.64 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.58742547692805 < 19.371847922576418.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 177.1773608910868 (T) = (0 -12.620563932195077) / Math.Sqrt((0.4702759466219753 / (299)) + (0.005268290004899419 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.37072262991511085 = (20.0556456217273 - 12.620563932195077) / 20.0556456217273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2123138677912237 < 18.152967770628205.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 212.83109453842263 (T) = (0 -2.1671744451865744) / Math.Sqrt((1.29144348403567 / (299)) + (0.05053168854533119 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8879873499976557 = (19.34758659081111 - 2.1671744451865744) / 19.34758659081111 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9748040366971744 < 18.743237141811594.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 368.5516818449449 (T) = (0 -2.1105981401846523) / Math.Sqrt((0.46690255569494926 / (299)) + (0.017905189747789468 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.8912130308163462 = (19.40120361862041 - 2.1105981401846523) / 19.40120361862041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2616466668777457 < 19.369340213797017.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 364.52218502041467 (T) = (0 -2.1249693401283944) / Math.Sqrt((0.5605197760050092 / (299)) + (0.015954365309736936 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.8945290508898107 = (20.14743735650244 - 2.1249693401283944) / 20.14743735650244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.432721491054506 < 17.322897246333593.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 37.914628848674006 (T) = (0 -12.302107855195219) / Math.Sqrt((0.25359962783423096 / (299)) + (0.6083281358504293 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.31649896520162596 = (17.998667491153423 - 12.302107855195219) / 17.998667491153423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1751052535616493 < 17.925582260765594.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 366.03534135293944 (T) = (0 -2.2905373059626397) / Math.Sqrt((0.4464918450061122 / (299)) + (0.013429639142746575 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.8765136419734875 = (18.548909714147218 - 2.2905373059626397) / 18.548909714147218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.31305524749448 < 18.170521444365225.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 34.31286594208429 (T) = (0 -12.593863871807345) / Math.Sqrt((0.8875304804376908 / (299)) + (0.8893631758168148 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.35730492452939244 = (19.595395005299526 - 12.593863871807345) / 19.595395005299526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.010366699140487 < 19.452620984992105.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 65.36458066627723 (T) = (0 -18.04547693346571) / Math.Sqrt((0.3914970765020964 / (299)) + (0.0006066460374103829 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.11672466181758476 = (20.4301831528539 - 18.04547693346571) / 20.4301831528539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.0336729079275 < 19.738680682397366.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 64.10260152091756 (T) = (0 -18.043901022867505) / Math.Sqrt((0.4041062973479435 / (299)) + (0.0006807612455505413 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11646486396829245 = (20.422392146066233 - 18.043901022867505) / 20.422392146066233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.297728794587725 < 18.15117823785723.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 33.108503789785814 (T) = (0 -12.460742776126866) / Math.Sqrt((0.8494757010215835 / (299)) + (0.9697341117994537 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.36047537364680926 = (19.484383028660982 - 12.460742776126866) / 19.484383028660982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.660188211141781 < 18.752274784254233.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 147.7017958407932 (T) = (0 -12.61892147673751) / Math.Sqrt((0.626288330921791 / (299)) + (0.0028114523358934267 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.35416426179020843 = (19.538902433173206 - 12.61892147673751) / 19.538902433173206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 31.87 μs 27.37 μs 0.86 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Get.GetString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.365478814270368 < 30.283674959758237.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 43.46631999138886 (T) = (0 -27441.870796521158) / Math.Sqrt((1348431.3694033716 / (299)) + (51447.06923784951 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.11512382322350398 = (31012.102615858406 - 27441.870796521158) / 31012.102615858406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 114.59 μs 108.48 μs 0.95 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.48227848768687 < 108.74948436944456.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 24.05055187267478 (T) = (0 -106459.58144908422) / Math.Sqrt((4873933.719401865 / (299)) + (3204901.740128067 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07552255195900266 = (115156.49373024306 - 106459.58144908422) / 115156.49373024306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteString - Duration of single invocation 8.08 secs 6.87 secs 0.85 0.00 False
WriteString - Duration of single invocation 360.37 ms 335.84 ms 0.93 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 100)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.870300426333333 < 7.671808131143333.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 146.27111348636143 (T) = (0 -6888096670.082354) / Math.Sqrt((15435328469244238 / (299)) + (168722675580397 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1401954471711074 = (8011235399.276998 - 6888096670.082354) / 8011235399.276998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 2)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 335.8437697333334 < 342.0899502019231.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 103.94495968696148 (T) = (0 -336016974.1133222) / Math.Sqrt((10560547474496.22 / (299)) + (263578654031.82394 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06271097187932714 = (358498781.093233 - 336016974.1133222) / 358498781.093233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.IO.Tests.Perf_Directory

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCurrentDirectory - Duration of single invocation 2.26 μs 2.08 μs 0.92 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Directory*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_Directory.GetCurrentDirectory

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0810117929426837 < 2.142075253183977.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 39.27701921744686 (T) = (0 -2068.8014587118805) / Math.Sqrt((2392.7632335368035 / (299)) + (210.7078332969432 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06960530637418057 = (2223.574008843067 - 2068.8014587118805) / 2223.574008843067 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in BilinearTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Interpol_Vector - Duration of single invocation 107.67 μs 101.69 μs 0.94 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BilinearTest*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

BilinearTest.Interpol_Vector

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 101.68721530032468 < 102.26839136696806.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 51.136493558682105 (T) = (0 -101769.28323388465) / Math.Sqrt((814556.4904629403 / (299)) + (317948.87740497256 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.05626755714427048 = (107837.00825832725 - 101769.28323388465) / 107837.00825832725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 41.49 μs 37.34 μs 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.3439459479665 < 39.302781520888445.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/17/2022 12:47:34 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 36.154525568851824 (T) = (0 -37146.724062011584) / Math.Sqrt((1227380.9135932198 / (299)) + (23373.963685068415 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.06511579600754959 = (39734.03754537237 - 37146.724062011584) / 39734.03754537237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 87.86 μs 78.78 μs 0.90 0.01 False
SerializeObjectProperty - Duration of single invocation 88.43 μs 79.50 μs 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 78.77979868090452 < 83.26121087574835.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 46.64196725733377 (T) = (0 -78977.80631669558) / Math.Sqrt((3659394.362591054 / (299)) + (143890.75902248363 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07434422961503581 = (85320.9247362549 - 78977.80631669558) / 85320.9247362549 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.49929763764153 < 83.93692938665735.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 44.26925104407731 (T) = (0 -79385.57897893172) / Math.Sqrt((3761687.681238385 / (299)) + (254293.6813009543 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07585078451528428 = (85901.25668969381 - 79385.57897893172) / 85901.25668969381 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAnyBenchmark - Duration of single invocation 11.96 ns 5.42 ns 0.45 0.02 False
LessThanOrEqualAllBenchmark - Duration of single invocation 12.18 ns 6.10 ns 0.50 0.02 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 12.86 ns 2.12 ns 0.16 0.02 False
MultiplyOperatorBenchmark - Duration of single invocation 14.30 ns 11.65 ns 0.82 0.01 False
LessThanAnyBenchmark - Duration of single invocation 11.98 ns 2.18 ns 0.18 0.01 False
MultiplyBenchmark - Duration of single invocation 13.86 ns 12.12 ns 0.87 0.00 False
DivideBenchmark - Duration of single invocation 17.19 ns 14.29 ns 0.83 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.4161344924683315 < 11.363600842230525.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 205.3886437427804 (T) = (0 -5.568667354539947) / Math.Sqrt((0.009268232579645258 / (299)) + (0.025681892476783846 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5317770714200051 = (11.893196626291555 - 5.568667354539947) / 11.893196626291555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.099796781343772 < 11.573942332865006.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 44.21585651126678 (T) = (0 -6.837964712478403) / Math.Sqrt((0.06605429891140317 / (299)) + (0.3145031553313929 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.4325337460184915 = (12.049993571426056 - 6.837964712478403) / 12.049993571426056 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.120423936332343 < 12.214976056016647.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 442.6009195771209 (T) = (0 -2.037822146649605) / Math.Sqrt((0.07107435078875024 / (299)) + (0.007972124307946245 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8400038568419356 = (12.736695437941814 - 2.037822146649605) / 12.736695437941814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.653935304999314 < 13.581044882661772.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 45.16361974701845 (T) = (0 -11.969567868075202) / Math.Sqrt((0.044895379774585235 / (299)) + (0.04332460099730406 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1454189091195109 = (14.006357027795639 - 11.969567868075202) / 14.006357027795639 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1789604245961693 < 11.370138046368131.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 387.6623413221037 (T) = (0 -2.0835672435412924) / Math.Sqrt((0.03687354623987745 / (299)) + (0.0118014149764623 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.8243690730602438 = (11.86332771708245 - 2.0835672435412924) / 11.86332771708245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.117380996612004 < 13.183249887691305.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 46.79574093707512 (T) = (0 -11.997849016902617) / Math.Sqrt((0.043175602096688526 / (299)) + (0.038919533369806164 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.14321163516147115 = (14.003281918007538 - 11.997849016902617) / 14.003281918007538 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.286446727454006 < 16.317603292674484.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 11:25:18 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 5.517604920882058 (T) = (0 -14.688474025158504) / Math.Sqrt((0.17651489015580046 / (299)) + (0.7353410723273442 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.126138039735819 = (16.808689121469445 - 14.688474025158504) / 16.808689121469445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 93.79 μs 85.49 μs 0.91 0.00 False
SerializeToString - Duration of single invocation 93.19 μs 85.39 μs 0.92 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.49000531648451 < 89.50060475056478.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 52.552470213285225 (T) = (0 -85194.05927229147) / Math.Sqrt((2525317.0594496583 / (299)) + (175276.43925667298 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06959819045065266 = (91566.9535440353 - 85194.05927229147) / 91566.9535440353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.39492634773015 < 88.76114737944657.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 10/25/2022 8:17:52 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 52.69672095823911 (T) = (0 -84549.10669082393) / Math.Sqrt((2447384.788514654 / (299)) + (188061.84016367912 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07070444842355543 = (90981.93416227588 - 84549.10669082393) / 90981.93416227588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Widen_sbyte - Duration of single invocation 30.12 μs 26.65 μs 0.88 0.00 False
Widen_float - Duration of single invocation 32.96 μs 29.99 μs 0.91 0.00 False
Widen_ushort - Duration of single invocation 23.60 μs 20.93 μs 0.89 0.01 False
Widen_short - Duration of single invocation 23.62 μs 20.72 μs 0.88 0.00 False
Widen_uint - Duration of single invocation 20.93 μs 17.47 μs 0.83 0.00 False
Widen_int - Duration of single invocation 20.97 μs 17.27 μs 0.82 0.00 False
Widen_byte - Duration of single invocation 30.04 μs 27.20 μs 0.91 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorConvert.Widen_sbyte

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.652819052174443 < 28.589689549698324.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 95.65910997625781 (T) = (0 -26784.329475391685) / Math.Sqrt((169847.80902021236 / (299)) + (11599.22885123339 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10470846446816054 = (29916.880046766786 - 26784.329475391685) / 29916.880046766786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_float

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.99316960726414 < 31.31386456577381.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 646.9366480525224 (T) = (0 -30007.764859464714) / Math.Sqrt((644.3307786322771 / (299)) + (429.6394074463488 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08959029616418115 = (32960.726069848926 - 30007.764859464714) / 32960.726069848926 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_ushort

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.932045281124495 < 22.401543923797867.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 19.613935513559216 (T) = (0 -21142.267830961093) / Math.Sqrt((30599.377454245332 / (299)) + (341631.31876082165 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.10189359281509172 = (23540.93864805062 - 21142.267830961093) / 23540.93864805062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_short

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.71841546357616 < 22.445857439336855.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 207.33462934426328 (T) = (0 -20718.415765939753) / Math.Sqrt((53510.90815737743 / (299)) + (271.9113862437914 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.12144025672356805 = (23582.250296006183 - 20718.415765939753) / 23582.250296006183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_uint

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.470913981119793 < 19.847665895928206.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 77.04977932334067 (T) = (0 -17862.32914558054) / Math.Sqrt((4389.204119536702 / (299)) + (43661.49390988377 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14612730610041927 = (20919.194715086218 - 17862.32914558054) / 20919.194715086218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_int

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.271285251614426 < 19.911144826098266.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 91.23989522082657 (T) = (0 -17526.4084296806) / Math.Sqrt((8613.850241581025 / (299)) + (31072.666275163087 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.16204002659263725 = (20915.56755200809 - 17526.4084296806) / 20915.56755200809 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Widen_byte

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.196195621345026 < 28.55773941167397.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 35.20980736553365 (T) = (0 -27493.608303514568) / Math.Sqrt((156243.67643360648 / (299)) + (96094.64364139947 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08071562382204464 = (29907.620553525372 - 27493.608303514568) / 29907.620553525372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 65.74 μs 61.65 μs 0.94 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 61.64611800478066 < 62.37514851215485.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 40.28640985379071 (T) = (0 -61726.66105738324) / Math.Sqrt((1540581.8898921206 / (299)) + (46166.75389675433 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.051075446044814725 = (65049.07139361302 - 61726.66105738324) / 65049.07139361302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 15, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualAllBenchmark - Duration of single invocation 8.67 ns 1.97 ns 0.23 0.03 False
LessThanAnyBenchmark - Duration of single invocation 8.94 ns 2.74 ns 0.31 0.05 False
EqualsAnyBenchmark - Duration of single invocation 8.89 ns 4.19 ns 0.47 0.01 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 9.32 ns 3.51 ns 0.38 0.03 False
LessThanOrEqualAllBenchmark - Duration of single invocation 8.69 ns 3.26 ns 0.38 0.03 False
LessThanAllBenchmark - Duration of single invocation 8.31 ns 4.19 ns 0.50 0.01 False
GreaterThanAllBenchmark - Duration of single invocation 8.49 ns 2.74 ns 0.32 0.04 False
GreaterThanAnyBenchmark - Duration of single invocation 9.09 ns 4.13 ns 0.45 0.01 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 9.32 ns 1.98 ns 0.21 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt64>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9663537006261431 < 8.23784354716107.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 226.47419665096123 (T) = (0 -2.073688108703096) / Math.Sqrt((0.027057863527066833 / (299)) + (0.020736262533896814 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.7589455587020647 = (8.602571674421409 - 2.073688108703096) / 8.602571674421409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.739024676989314 < 8.492955537230834.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 161.91201588700898 (T) = (0 -2.3522294649684206) / Math.Sqrt((0.006392365777895408 / (299)) + (0.04509216467481317 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.7354899963819428 = (8.892780737188883 - 2.3522294649684206) / 8.892780737188883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.190013465155051 < 8.477279743987095.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 310.16236818603437 (T) = (0 -4.1816158386177396) / Math.Sqrt((0.0052508668569487904 / (299)) + (0.005934281466535205 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5291156586846975 = (8.88034591878209 - 4.1816158386177396) / 8.88034591878209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.507333005532968 < 8.866249643243268.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 246.04423258104876 (T) = (0 -3.354680353409118) / Math.Sqrt((0.027766510695193484 / (299)) + (0.013477516554120675 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.6373526700554665 = (9.250530960540127 - 3.354680353409118) / 9.250530960540127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.264720183099151 < 8.24208820797783.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 191.99893755494497 (T) = (0 -3.34778090430805) / Math.Sqrt((0.03573435365508289 / (299)) + (0.016777341642854396 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.6095371153668263 = (8.573877405667313 - 3.34778090430805) / 8.573877405667313 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.185460591005539 < 7.908730726416832.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 282.70636817755195 (T) = (0 -4.19283634461568) / Math.Sqrt((0.009663024984010884 / (299)) + (0.004901182286853582 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.49262662533537666 = (8.263808378567694 - 4.19283634461568) / 8.263808378567694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.736382645616916 < 8.079575928674995.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 192.59358533272047 (T) = (0 -2.4505969586703094) / Math.Sqrt((0.02638913547705421 / (299)) + (0.024524390725279645 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.709324967909747 = (8.430710202551598 - 2.4505969586703094) / 8.430710202551598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.130313506982144 < 8.67131731968044.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 315.5457056907116 (T) = (0 -4.181665757916005) / Math.Sqrt((0.027858921767802955 / (299)) + (0.004037324956387107 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5375878607159267 = (9.043157397187374 - 4.181665757916005) / 9.043157397187374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9768544326891966 < 8.856421402480013.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 287.66479592979664 (T) = (0 -2.095418094580787) / Math.Sqrt((0.03648991238092063 / (299)) + (0.0137529596104333 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.7727047043771815 = (9.218924170159662 - 2.095418094580787) / 9.218924170159662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare 803688595257517f4b47e873d11152b0560c204a
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 124.90 μs 117.78 μs 0.94 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.7815623433584 < 119.70677867775537.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 33.058179466364166 (T) = (0 -117532.95330335095) / Math.Sqrt((9814603.812482083 / (299)) + (205094.8649515745 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.054342476781077446 = (124287.01767557526 - 117532.95330335095) / 124287.01767557526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAllText - Duration of single invocation 680.08 μs 518.09 μs 0.76 0.01 False
WriteAllText - Duration of single invocation 86.63 μs 69.12 μs 0.80 0.01 False
WriteAllTextAsync - Duration of single invocation 107.02 μs 88.61 μs 0.83 0.01 False
AppendAllTextAsync - Duration of single invocation 59.26 μs 50.98 μs 0.86 0.00 False
AppendAllText - Duration of single invocation 50.97 μs 43.37 μs 0.85 0.00 False
WriteAllTextAsync - Duration of single invocation 835.88 μs 645.81 μs 0.77 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_File.WriteAllText(size: 100000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 518.0890818055555 < 647.8234679299301.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 29.015391652947812 (T) = (0 -540582.8937186318) / Math.Sqrt((159034402.00827998 / (299)) + (606025082.3110157 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.20176098000634649 = (677219.3292717384 - 540582.8937186318) / 677219.3292717384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.11707812500002 < 81.28758780262085.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 28.376780023738707 (T) = (0 -71595.81420855784) / Math.Sqrt((1723518.2436911513 / (299)) + (6356053.389325185 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.16052515858855085 = (85286.43227494511 - 71595.81420855784) / 85286.43227494511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllTextAsync(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.60858321360153 < 101.7907587920228.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 27.308925705019185 (T) = (0 -91961.31251592097) / Math.Sqrt((2057957.0507662252 / (299)) + (8180977.267898218 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1397090057245456 = (106895.5889668143 - 91961.31251592097) / 106895.5889668143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.AppendAllTextAsync(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.98329375 < 55.766788390659336.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 66.89756141847603 (T) = (0 -51149.72490447246) / Math.Sqrt((973871.9448093527 / (299)) + (223756.80096716178 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.12181446314685512 = (58244.78172091099 - 51149.72490447246) / 58244.78172091099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.AppendAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.37200676666667 < 48.18911246514881.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 97.06918130474946 (T) = (0 -43570.409678365446) / Math.Sqrt((513654.2130808864 / (299)) + (92754.80419591138 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.13645343035621307 = (50455.193975628026 - 43570.409678365446) / 50455.193975628026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllTextAsync(size: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 645.8077291666665 < 796.3699209002978.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 19.601294662756896 (T) = (0 -674287.4618470637) / Math.Sqrt((11278670925.5225 / (299)) + (858435057.277374 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.19379722889777215 = (836374.5276206238 - 674287.4618470637) / 836374.5276206238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 1.29 μs 787.58 ns 0.61 0.01 False
GetString - Duration of single invocation 3.47 μs 2.73 μs 0.79 0.01 False
GetString - Duration of single invocation 2.53 μs 2.09 μs 0.83 0.01 False
GetChars - Duration of single invocation 2.53 μs 2.10 μs 0.83 0.01 False
GetChars - Duration of single invocation 3.46 μs 2.73 μs 0.79 0.01 False
GetChars - Duration of single invocation 107.18 ns 94.90 ns 0.89 0.01 False
GetByteCount - Duration of single invocation 78.59 ns 67.69 ns 0.86 0.01 False
GetChars - Duration of single invocation 146.52 ns 137.78 ns 0.94 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 787.577495921422 < 1.221369313963407.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 26.31978519473894 (T) = (0 -910.6967805581534) / Math.Sqrt((844.3747588641631 / (299)) + (5183.187926609745 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.28377079300951386 = (1271.5158383233181 - 910.6967805581534) / 1271.5158383233181 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.730919769182782 < 3.2775594311080805.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 63.39558462857713 (T) = (0 -2754.078514207929) / Math.Sqrt((20287.49114520339 / (299)) + (480.3238883063257 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.17819598397337025 = (3351.25949800504 - 2754.078514207929) / 3351.25949800504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0907046881862623 < 2.400577018246693.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 48.09124387559545 (T) = (0 -2089.488262047839) / Math.Sqrt((13229.397166893003 / (299)) + (248.9036326241835 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.14587896395278602 = (2446.3608480102303 - 2089.488262047839) / 2446.3608480102303 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.096820353984586 < 2.4044757541611963.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 38.80279461303633 (T) = (0 -2092.9395009920795) / Math.Sqrt((9872.982773518643 / (299)) + (1520.91265153369 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.15583732666628522 = (2479.3082744665467 - 2092.9395009920795) / 2479.3082744665467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7334235508913456 < 3.2813357007605655.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 65.8005641940868 (T) = (0 -2746.2848496694855) / Math.Sqrt((14654.859166204442 / (299)) + (1057.0070959821949 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1892951283309484 = (3387.5272563929802 - 2746.2848496694855) / 3387.5272563929802 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 94.89674995572074 < 102.00759909398423.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 33.98747189629832 (T) = (0 -94.60352890195408) / Math.Sqrt((20.241065203414404 / (299)) + (0.5888177702305543 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.09888383751841869 = (104.98483196819562 - 94.60352890195408) / 104.98483196819562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.6926297878688 < 74.67406704525754.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 56.62505798555675 (T) = (0 -68.41015915442136) / Math.Sqrt((5.460825447801921 / (299)) + (0.17832127023844965 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.11497317136564449 = (77.29727160924823 - 68.41015915442136) / 77.29727160924823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 137.78155573197276 < 138.8289930012066.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 22.442326158401045 (T) = (0 -136.3828609873935) / Math.Sqrt((21.494142226717823 / (299)) + (0.8917954107999833 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.05189899506698187 = (143.84845103821903 - 136.3828609873935) / 143.84845103821903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 6.48 μs 5.56 μs 0.86 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.557267926428571 < 6.151294432778845.
IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 9/13/2022 4:57:27 AM, 11/2/2022 9:35:40 AM, 11/9/2022 12:33:05 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 12:33:05 PM.
IsImprovementStdDev: Marked as improvement because 16.878178918201517 (T) = (0 -5593.628268432497) / Math.Sqrt((147566.67550958032 / (299)) + (29975.863518639842 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10661163778616757 = (6261.138498123465 - 5593.628268432497) / 6261.138498123465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added Improvement PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Nov 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-jit os-linux perf-improvement PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos