Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 116 Regressions on 9/28/2022 9:30:44 PM #8870

Closed
performanceautofiler bot opened this issue Oct 4, 2022 · 20 comments
Closed

[Perf] Linux/x64: 116 Regressions on 9/28/2022 9:30:44 PM #8870

performanceautofiler bot opened this issue Oct 4, 2022 · 20 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement perf-regression runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 32.42 ns 39.66 ns 1.22 0.19 False
EqualsStaticBenchmark - Duration of single invocation 46.76 ns 51.34 ns 1.10 0.16 False
EqualsAnyBenchmark - Duration of single invocation 78.95 ns 85.99 ns 1.09 0.28 False
SubtractBenchmark - Duration of single invocation 49.77 ns 54.49 ns 1.09 0.21 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 63.75 ns 71.89 ns 1.13 0.30 False
NegateBenchmark - Duration of single invocation 48.78 ns 55.01 ns 1.13 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int64>.DotBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.66201913635898 > 34.02479590576788.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.7382527986333125 (T) = (0 -36.018782313399505) / Math.Sqrt((0.3034650856593724 / (35)) + (4.908809239675121 / (27))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (27) - 2, .025) and -0.1034654218234154 = (32.64151427045214 - 36.018782313399505) / 32.64151427045214 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.339459933956654 > 49.25958606036935.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.049067269281117 (T) = (0 -50.607425728921356) / Math.Sqrt((5.66998843602138 / (34)) + (2.749492341661619 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.07620255622229787 = (47.02407129245659 - 50.607425728921356) / 47.02407129245659 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.99197152416185 > 82.85673454955499.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.432425698687042 (T) = (0 -85.0674550832774) / Math.Sqrt((13.236275702035277 / (38)) + (12.447952550342263 / (26))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (26) - 2, .025) and -0.061658310667565386 = (80.12696196932458 - 85.0674550832774) / 80.12696196932458 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.SubtractBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.48551080072452 > 52.21634013593077.
IsChangePoint: Marked as a change because one of 7/22/2022 4:05:57 PM, 7/23/2022 2:45:25 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.622431101009044 (T) = (0 -54.51933054616863) / Math.Sqrt((1.252354649202479 / (35)) + (8.133284829805143 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.09602681233829041 = (49.7426978358821 - 54.51933054616863) / 49.7426978358821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.88853966211951 > 66.37716252903661.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.926176197572709 (T) = (0 -70.05111198010887) / Math.Sqrt((14.706379437964136 / (35)) + (4.5953957622733 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.09370514705448933 = (64.04935751538423 - 70.05111198010887) / 64.04935751538423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.0119912855032 > 51.188679882862345.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.856585139016438 (T) = (0 -53.35482797036031) / Math.Sqrt((4.159235724572628 / (35)) + (4.771010827993788 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.08415323524008453 = (49.213364159306266 - 53.35482797036031) / 49.213364159306266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OneBenchmark - Duration of single invocation 45.07 ns 48.26 ns 1.07 0.16 False
SquareRootBenchmark - Duration of single invocation 74.23 ns 85.05 ns 1.15 0.12 False
AddBenchmark - Duration of single invocation 76.46 ns 83.82 ns 1.10 0.21 False
MinBenchmark - Duration of single invocation 89.84 ns 109.24 ns 1.22 0.10 False
MultiplyOperatorBenchmark - Duration of single invocation 77.25 ns 84.61 ns 1.10 0.18 False
LessThanOrEqualBenchmark - Duration of single invocation 80.61 ns 91.92 ns 1.14 0.11 False
GreaterThanBenchmark - Duration of single invocation 78.43 ns 85.21 ns 1.09 0.19 False
UnaryNegateOperatorBenchmark - Duration of single invocation 78.33 ns 83.13 ns 1.06 0.20 False
MaxBenchmark - Duration of single invocation 89.65 ns 96.27 ns 1.07 0.14 False
EqualsStaticBenchmark - Duration of single invocation 78.19 ns 86.26 ns 1.10 0.15 False
OnesComplementBenchmark - Duration of single invocation 64.72 ns 75.03 ns 1.16 0.22 False
OnesComplementOperatorBenchmark - Duration of single invocation 64.75 ns 71.10 ns 1.10 0.24 False
GreaterThanOrEqualBenchmark - Duration of single invocation 83.95 ns 90.00 ns 1.07 0.16 False
LessThanBenchmark - Duration of single invocation 79.45 ns 87.65 ns 1.10 0.17 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 99.57 ns 114.44 ns 1.15 0.27 False
SubtractionOperatorBenchmark - Duration of single invocation 73.26 ns 83.87 ns 1.14 0.23 False
NegateBenchmark - Duration of single invocation 78.63 ns 86.48 ns 1.10 0.17 False
LessThanAnyBenchmark - Duration of single invocation 92.89 ns 109.09 ns 1.17 0.21 False
GreaterThanAnyBenchmark - Duration of single invocation 109.31 ns 126.94 ns 1.16 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt32>.OneBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.258819275265964 > 44.715945829797974.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -24.044836899520227 (T) = (0 -48.53685474209878) / Math.Sqrt((0.9361315138069796 / (35)) + (0.7110773533396154 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.127550122814257 = (43.0462945815264 - 48.53685474209878) / 43.0462945815264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.SquareRootBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.04722190708057 > 78.99928819618886.
IsChangePoint: Marked as a change because one of 7/27/2022 5:17:06 PM, 7/28/2022 6:38:38 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.974325746039838 (T) = (0 -81.10260780753927) / Math.Sqrt((4.781039588252283 / (35)) + (15.771869599677824 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07635241632531435 = (75.34949202271963 - 81.10260780753927) / 75.34949202271963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.AddBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.81946615394388 > 80.7017430405793.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.607196143502355 (T) = (0 -88.40443467509938) / Math.Sqrt((10.389335119860284 / (35)) + (37.20708813520844 / (31))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (31) - 2, .025) and -0.13523673376174777 = (77.87312729227878 - 88.40443467509938) / 77.87312729227878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.24139237936808 > 94.82814979058425.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/3/2022 11:00:05 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -2.687075572851525 (T) = (0 -104.38453828881063) / Math.Sqrt((27.206252641769716 / (60)) + (57.90193727731856 / (4))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (60) + (4) - 2, .025) and -0.11044751964441218 = (94.00222562722881 - 104.38453828881063) / 94.00222562722881 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.60871499503284 > 79.74653224136745.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.193348145746184 (T) = (0 -86.67338650889964) / Math.Sqrt((27.377803400796253 / (35)) + (36.93908193491971 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.13676616090792437 = (76.24557229929718 - 86.67338650889964) / 76.24557229929718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.91823020526772 > 86.38117188984734.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.416118992396486 (T) = (0 -93.23383978626877) / Math.Sqrt((3.146028191812109 / (36)) + (32.19172697217373 / (27))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (27) - 2, .025) and -0.12898410110466033 = (82.58206620894275 - 93.23383978626877) / 82.58206620894275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.2089728689246 > 82.25942088576373.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.69481279277953 (T) = (0 -88.25562721906003) / Math.Sqrt((5.180440193533733 / (35)) + (29.524431889281608 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.11905158153000207 = (78.8664514448871 - 88.25562721906003) / 78.8664514448871 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.13395807120835 > 81.67118067070355.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.108816063120488 (T) = (0 -88.6037384832415) / Math.Sqrt((5.568206045138926 / (35)) + (46.55097123506805 / (27))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (27) - 2, .025) and -0.16424963311994972 = (76.10372892779162 - 88.6037384832415) / 76.10372892779162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.MaxBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.26739073191526 > 94.47533741867284.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.5931909821919135 (T) = (0 -98.36425509187394) / Math.Sqrt((15.07697330019848 / (35)) + (20.582982181857492 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.0771025050160379 = (91.32302137799718 - 98.36425509187394) / 91.32302137799718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.26491293640039 > 81.89559101040784.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.091997432888766 (T) = (0 -87.25920018388334) / Math.Sqrt((2.01611168070183 / (35)) + (21.645235631436353 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.11369222673208232 = (78.35126984762104 - 87.25920018388334) / 78.35126984762104 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.0283824063584 > 67.90721194749811.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -12.294204068767328 (T) = (0 -72.66776288665437) / Math.Sqrt((2.600008943393378 / (35)) + (7.705861515319054 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.10785839727782669 = (65.5930063491958 - 72.66776288665437) / 65.5930063491958 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.10140525274325 > 68.40374879748938.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.465354132331976 (T) = (0 -72.08213121052206) / Math.Sqrt((3.8649514204625017 / (35)) + (8.18722462695919 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.08849633165764328 = (66.22174931977035 - 72.08213121052206) / 66.22174931977035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.00214881130944 > 87.98307316294543.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.322066725115267 (T) = (0 -92.2191990555902) / Math.Sqrt((15.380242043094007 / (36)) + (31.775617394950956 / (29))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (29) - 2, .025) and -0.09242064832294375 = (84.41729767481304 - 92.2191990555902) / 84.41729767481304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.65044817049666 > 83.4410747877965.
IsChangePoint: Marked as a change because one of 8/3/2022 8:57:58 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.585836614258335 (T) = (0 -89.2861582537044) / Math.Sqrt((1.4613388704359778 / (34)) + (22.452587195606075 / (31))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (31) - 2, .025) and -0.12823727984824712 = (79.13774863538792 - 89.2861582537044) / 79.13774863538792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.44264580822069 > 103.73840886981877.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.656723504087495 (T) = (0 -113.6964703826205) / Math.Sqrt((33.61103248118534 / (35)) + (32.401724775153475 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.13950204376264624 = (99.77732905786962 - 113.6964703826205) / 99.77732905786962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.87332687202547 > 79.28800346706112.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.8116621571169 (T) = (0 -87.56456828952933) / Math.Sqrt((12.847941153038139 / (35)) + (79.28544350580991 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.1587314177297264 = (75.56933983985039 - 87.56456828952933) / 75.56933983985039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.48127420298087 > 79.53947822682177.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.394232819338436 (T) = (0 -88.69863141049231) / Math.Sqrt((7.427439129659963 / (35)) + (39.98565421754405 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.1569314190867068 = (76.66714720265087 - 88.69863141049231) / 76.66714720265087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.08873810986577 > 99.51940855608618.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -13.42374436860692 (T) = (0 -109.95621503775772) / Math.Sqrt((5.199397139044979 / (35)) + (33.86262415073771 / (32))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (32) - 2, .025) and -0.15488250508200777 = (95.20987161369266 - 109.95621503775772) / 95.20987161369266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.94060022668842 > 114.88409966045221.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/29/2022 2:35:34 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.418882857473922 (T) = (0 -121.8256093521611) / Math.Sqrt((12.464268653120662 / (39)) + (35.71243219467014 / (25))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (25) - 2, .025) and -0.08756652910739585 = (112.01669607481178 - 121.8256093521611) / 112.01669607481178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 430.57 μs 470.07 μs 1.09 0.05 False
ImmutableSortedSet - Duration of single invocation 507.75 μs 538.73 μs 1.06 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 470.071274215368 > 459.064705068109.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -13.84574285486324 (T) = (0 -471279.6671298911) / Math.Sqrt((46780279.45062267 / (35)) + (145621026.83207983 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.08121774756380258 = (435878.5898509134 - 471279.6671298911) / 435878.5898509134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 538.7266596485412 > 535.2512622103495.
IsChangePoint: Marked as a change because one of 8/2/2022 2:18:01 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -21.393681872493534 (T) = (0 -545552.1197715452) / Math.Sqrt((31399792.109588344 / (35)) + (50743990.71394843 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.06734193758477452 = (511131.5320430893 - 545552.1197715452) / 511131.5320430893 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OneBenchmark - Duration of single invocation 34.66 ns 38.49 ns 1.11 0.21 False
MaxBenchmark - Duration of single invocation 58.27 ns 62.34 ns 1.07 0.19 False
LessThanOrEqualBenchmark - Duration of single invocation 56.57 ns 61.78 ns 1.09 0.27 False
AndNotBenchmark - Duration of single invocation 83.73 ns 88.53 ns 1.06 0.29 False
MultiplyBenchmark - Duration of single invocation 55.08 ns 61.49 ns 1.12 0.24 False
SquareRootBenchmark - Duration of single invocation 35.35 ns 38.44 ns 1.09 0.21 False
DivideBenchmark - Duration of single invocation 56.04 ns 61.91 ns 1.10 0.22 False
OnesComplementBenchmark - Duration of single invocation 60.62 ns 64.71 ns 1.07 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Double>.OneBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.48547492244615 > 36.437204675782716.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -17.22652502412602 (T) = (0 -38.866738279654) / Math.Sqrt((0.8650238963454278 / (35)) + (0.7138671586396629 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.10818026794719117 = (35.07257745317131 - 38.866738279654) / 35.07257745317131 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.MaxBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.341376186331125 > 61.398367734998565.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.526485542753173 (T) = (0 -62.43185769274923) / Math.Sqrt((6.3767971085968815 / (35)) + (4.365339465037886 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.053807860466580364 = (59.24406149818157 - 62.43185769274923) / 59.24406149818157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.77626692891586 > 59.32879982365056.
IsChangePoint: Marked as a change because one of 7/6/2022 12:32:56 PM, 7/7/2022 1:00:04 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.22261491467451 (T) = (0 -62.10827326245255) / Math.Sqrt((1.5729644122168314 / (35)) + (6.845692460512102 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.094115625328189 = (56.76573099284882 - 62.10827326245255) / 56.76573099284882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AndNotBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 88.52881760057112 > 88.25894660627846.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 11:19:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.976930503355394 (T) = (0 -89.6600879607903) / Math.Sqrt((9.174689439498968 / (33)) + (14.182841213611896 / (31))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (31) - 2, .025) and -0.060638313313575345 = (84.5340837072727 - 89.6600879607903) / 84.5340837072727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.MultiplyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.48585851137682 > 58.07436209575573.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.955856257283941 (T) = (0 -60.35372995243013) / Math.Sqrt((5.75305272713782 / (35)) + (12.115476835621358 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.08254651681132534 = (55.7516273113177 - 60.35372995243013) / 55.7516273113177 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.SquareRootBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.43862020884357 > 37.02715693967304.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.571803790044796 (T) = (0 -39.09720246008806) / Math.Sqrt((0.6733158458005493 / (35)) + (5.9358834305574435 / (27))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (27) - 2, .025) and -0.10460087942329568 = (35.3948681269387 - 39.09720246008806) / 35.3948681269387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.DivideBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.91453747902374 > 59.04533209015445.
IsChangePoint: Marked as a change because one of 9/27/2022 4:04:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.968352647970752 (T) = (0 -59.79498890806816) / Math.Sqrt((3.5950896942478283 / (27)) + (9.263371295025019 / (37))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (37) - 2, .025) and -0.05424755785634824 = (56.71816686931878 - 59.79498890806816) / 56.71816686931878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.71413367785841 > 62.994042557395254.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.585426715304845 (T) = (0 -65.24126813954976) / Math.Sqrt((2.765003012084519 / (35)) + (1.7718371153659143 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07119930643531491 = (60.904882730606396 - 65.24126813954976) / 60.904882730606396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TransformByQuaternionBenchmark - Duration of single invocation 97.97 ns 110.31 ns 1.13 0.15 False
TransformVector2ByQuaternionBenchmark - Duration of single invocation 82.53 ns 97.99 ns 1.19 0.16 False
DistanceSquaredJitOptimizeCanaryBenchmark - Duration of single invocation 41.30 ns 43.97 ns 1.06 0.21 False
TransformVector3ByQuaternionBenchmark - Duration of single invocation 100.51 ns 108.40 ns 1.08 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.31150246736269 > 102.30289451780108.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -17.385997159763267 (T) = (0 -110.71597352728367) / Math.Sqrt((10.569591849417902 / (35)) + (5.477583578397243 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.12362112676294743 = (98.53496956420403 - 110.71597352728367) / 98.53496956420403 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.98974222652413 > 88.33612609499221.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -19.410575349532436 (T) = (0 -99.00418670103376) / Math.Sqrt((1.3514518621085845 / (35)) + (17.568301778158975 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.18678408061367424 = (83.42224025270012 - 99.00418670103376) / 83.42224025270012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.966419462708515 > 43.29864176654762.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.656290133237657 (T) = (0 -44.593587768686135) / Math.Sqrt((5.035542923585861 / (38)) + (4.150414153027851 / (26))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (26) - 2, .025) and -0.05981313881350048 = (42.07683990274954 - 44.593587768686135) / 42.07683990274954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.39647505785852 > 103.58246817439989.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -18.381704917240988 (T) = (0 -110.80551630745937) / Math.Sqrt((7.3134360640430085 / (34)) + (8.656663369022134 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.13344143989538354 = (97.76024804394544 - 110.80551630745937) / 97.76024804394544 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinkedList - Duration of single invocation 24.12 ms 25.88 ms 1.07 0.05 False
ImmutableList - Duration of single invocation 33.03 ms 35.29 ms 1.07 0.03 False
SortedSet - Duration of single invocation 484.20 μs 530.88 μs 1.10 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<Int32>.LinkedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.880245548148153 > 25.284234717807696.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -21.491963924699846 (T) = (0 -26159701.30816579) / Math.Sqrt((140080333978.49927 / (35)) + (147613668211.14514 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.0841348680561399 = (24129563.65389325 - 26159701.30816579) / 24129563.65389325 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.28758488571429 > 34.5770359739011.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -24.61463717176828 (T) = (0 -35366075.84638671) / Math.Sqrt((43674106135.44429 / (35)) + (271748358231.89142 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.07602874636735006 = (32867222.149763025 - 35366075.84638671) / 32867222.149763025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.SortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 530.8812597701149 > 509.8070989746094.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -12.916082227670486 (T) = (0 -527995.6843890157) / Math.Sqrt((80298684.71424282 / (35)) + (175037473.07062998 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.07497432338634782 = (491170.5078924504 - 527995.6843890157) / 491170.5078924504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 558.93 μs 610.59 μs 1.09 0.10 False
ImmutableSortedDictionary - Duration of single invocation 533.56 μs 592.32 μs 1.11 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueTrue<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 610.5867367788461 > 583.5477222656251.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -18.56608005946948 (T) = (0 -603826.2070683208) / Math.Sqrt((157716428.3132816 / (35)) + (68557317.93377624 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.08765683029237484 = (555162.4283056314 - 603826.2070683208) / 555162.4283056314 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;.ImmutableSortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 592.3173149691359 > 559.0295031291446.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.688440306013263 (T) = (0 -569056.7574460427) / Math.Sqrt((48019206.20968566 / (35)) + (92003307.45796165 / (31))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (31) - 2, .025) and -0.06506898925179325 = (534290.9832026964 - 569056.7574460427) / 534290.9832026964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 589.01 μs 622.98 μs 1.06 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 622.9836670833333 > 621.1781059945913.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.94488663698173 (T) = (0 -632724.2995194475) / Math.Sqrt((157033385.60357687 / (35)) + (100465945.6511378 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.05668625794475149 = (598781.6106836598 - 632724.2995194475) / 598781.6106836598 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanBenchmark - Duration of single invocation 78.00 ns 84.05 ns 1.08 0.21 False
EqualsStaticBenchmark - Duration of single invocation 79.11 ns 84.22 ns 1.06 0.15 False
UnaryNegateOperatorBenchmark - Duration of single invocation 75.73 ns 95.08 ns 1.26 0.22 False
AddOperatorBenchmark - Duration of single invocation 74.65 ns 85.75 ns 1.15 0.18 False
EqualsAnyBenchmark - Duration of single invocation 111.13 ns 117.29 ns 1.06 0.27 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 96.57 ns 106.20 ns 1.10 0.23 False
LessThanBenchmark - Duration of single invocation 78.19 ns 87.50 ns 1.12 0.19 False
LessThanAllBenchmark - Duration of single invocation 136.63 ns 148.63 ns 1.09 0.21 False
LessThanOrEqualBenchmark - Duration of single invocation 82.62 ns 91.57 ns 1.11 0.16 False
AndNotBenchmark - Duration of single invocation 87.46 ns 96.46 ns 1.10 0.20 False
SubtractionOperatorBenchmark - Duration of single invocation 75.51 ns 82.23 ns 1.09 0.19 False
NegateBenchmark - Duration of single invocation 74.67 ns 83.23 ns 1.11 0.21 False
MinBenchmark - Duration of single invocation 89.79 ns 96.94 ns 1.08 0.11 False
GreaterThanOrEqualBenchmark - Duration of single invocation 82.48 ns 89.83 ns 1.09 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.04825874487139 > 81.79477112963359.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.289494993560247 (T) = (0 -87.82425825411748) / Math.Sqrt((7.938100407115147 / (35)) + (29.476225223696826 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.11581971400575554 = (78.70828696764222 - 87.82425825411748) / 78.70828696764222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.21871111139286 > 81.8142523839403.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.146930258402326 (T) = (0 -86.7966785432983) / Math.Sqrt((5.592651042382101 / (35)) + (23.932938489544462 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.10272868724802627 = (78.71081939466761 - 86.7966785432983) / 78.71081939466761 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.07847747653074 > 79.49652811130221.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/3/2022 11:00:05 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -2.3958903211139964 (T) = (0 -94.74266346480239) / Math.Sqrt((38.41875855072349 / (60)) + (133.65924375351167 / (4))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (60) + (4) - 2, .025) and -0.1731236331689067 = (80.76102192986961 - 94.74266346480239) / 80.76102192986961 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.75052123054584 > 78.639863287054.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.150604586103308 (T) = (0 -85.75217210395901) / Math.Sqrt((1.7642206936269202 / (35)) + (33.01154288500403 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.14818382382218656 = (74.6850550624366 - 85.75217210395901) / 74.6850550624366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.2919812794136 > 114.88670242990199.
IsChangePoint: Marked as a change because one of 7/29/2022 12:14:05 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.816066606758769 (T) = (0 -121.84060837297538) / Math.Sqrt((6.154944345091911 / (35)) + (42.04159624512881 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.10340156134701262 = (110.4227260873499 - 121.84060837297538) / 110.4227260873499 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.20331756865387 > 103.06360133834183.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.950396216815722 (T) = (0 -112.36409066443473) / Math.Sqrt((8.65885701821634 / (35)) + (58.28864157927188 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.13364357060293688 = (99.11765353608722 - 112.36409066443473) / 99.11765353608722 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.49824140003146 > 82.73281904429206.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.48887450873128 (T) = (0 -88.98357261127107) / Math.Sqrt((11.186191365932288 / (35)) + (24.7749199404252 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.11527650295040431 = (79.78610898361957 - 88.98357261127107) / 79.78610898361957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.63447026785437 > 143.9099370217615.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.6688453960592815 (T) = (0 -151.69695792414424) / Math.Sqrt((35.01359015476745 / (35)) + (63.74388002305515 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.09939810117116012 = (137.98182638531523 - 151.69695792414424) / 137.98182638531523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.5745286832234 > 85.65017682937027.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.988782481765686 (T) = (0 -92.80966096592914) / Math.Sqrt((0.4459024027580955 / (34)) + (37.628657786815104 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.13786988087140017 = (81.56438844734507 - 92.80966096592914) / 81.56438844734507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.AndNotBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.45910178255502 > 94.39210036535316.
IsChangePoint: Marked as a change because one of 7/29/2022 4:16:32 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.792826259790686 (T) = (0 -96.65333894913459) / Math.Sqrt((14.629863518124658 / (35)) + (7.553448350273927 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.0711351892190464 = (90.2344913340057 - 96.65333894913459) / 90.2344913340057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.23097945678008 > 78.88688643138845.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.947148440944098 (T) = (0 -86.02391979782509) / Math.Sqrt((24.92966648924343 / (35)) + (44.98613893001288 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.13830518388873916 = (75.57193010748274 - 86.02391979782509) / 75.57193010748274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.22534438630746 > 79.62356722753819.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.079342960143991 (T) = (0 -87.791415094701) / Math.Sqrt((7.403430136638723 / (35)) + (42.71931756086597 / (31))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (31) - 2, .025) and -0.1499399004013672 = (76.34435074742504 - 87.791415094701) / 76.34435074742504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.93540727078106 > 94.43107990593515.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.834559281562181 (T) = (0 -101.94899103144424) / Math.Sqrt((4.74909347669622 / (35)) + (53.064410491264915 / (27))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (27) - 2, .025) and -0.1253538192392796 = (90.5928333724944 - 101.94899103144424) / 90.5928333724944 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 89.82641539717098 > 87.44982689896966.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/29/2022 2:35:34 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.3450723857157465 (T) = (0 -91.12469177089234) / Math.Sqrt((7.918038558156717 / (39)) + (23.074467388249165 / (28))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (28) - 2, .025) and -0.0759263524356956 = (84.69417220296084 - 91.12469177089234) / 84.69417220296084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AnyWithPredicate_LastElementMatches - Duration of single invocation 59.57 μs 64.43 μs 1.08 0.01 False
FirstWithPredicate_LastElementMatches - Duration of single invocation 59.31 μs 64.03 μs 1.08 0.01 False
OrderBy - Duration of single invocation 64.36 μs 68.47 μs 1.06 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.43473470433256 > 62.12953774174528.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -17.409684421825997 (T) = (0 -63391.44499036366) / Math.Sqrt((292177.4042100101 / (35)) + (1315962.6263558913 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.06798572557099283 = (59356.07889934275 - 63391.44499036366) / 59356.07889934275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.03472652116402 > 61.90115034286668.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.956437720301484 (T) = (0 -63465.40606978437) / Math.Sqrt((351273.57847738767 / (35)) + (1468602.8513473982 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07044966882636956 = (59288.54753102704 - 63465.40606978437) / 59288.54753102704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.46744236111111 > 67.42321104616204.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -13.791416563902487 (T) = (0 -68593.60577057979) / Math.Sqrt((454935.1237820221 / (35)) + (2014622.1513445298 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.06224872148960301 = (64573.95935896277 - 68593.60577057979) / 64573.95935896277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline a7d80c177a7ea412fb00867ccc4f4b2c44c9e030
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 1.20 ms 1.27 ms 1.06 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndRemove<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2695404924450548 > 1.2553649996847265.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -20.51750815625915 (T) = (0 -1274542.7137187622) / Math.Sqrt((242935929.55525702 / (34)) + (264067643.4532685 / (31))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (31) - 2, .025) and -0.06804629814021522 = (1193340.3223606676 - 1274542.7137187622) / 1193340.3223606676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in Devirtualization.EqualityComparer

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ValueTupleCompareNoOpt - Duration of single invocation 266.38 ns 291.15 ns 1.09 0.06 False
ValueTupleCompareCached - Duration of single invocation 271.01 ns 290.25 ns 1.07 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Devirtualization.EqualityComparer*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Devirtualization.EqualityComparer.ValueTupleCompareNoOpt

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 291.14660442512866 > 283.55753250108836.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -12.507199091286619 (T) = (0 -295.77899287484576) / Math.Sqrt((41.799719709969835 / (35)) + (59.305896977435445 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.08337573046295499 = (273.0160779478156 - 295.77899287484576) / 273.0160779478156 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Devirtualization.EqualityComparer.ValueTupleCompareCached

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 290.2460050212453 > 282.70901758793855.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.42960843141698 (T) = (0 -291.59993198561966) / Math.Sqrt((24.202989273281418 / (35)) + (35.34590624415349 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.08445101619759808 = (268.8917504158502 - 291.59993198561966) / 268.8917504158502 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 561.55 μs 605.67 μs 1.08 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSet<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 605.6685273008243 > 587.271948733259.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -15.491321884123009 (T) = (0 -608285.0777751901) / Math.Sqrt((184266130.24009365 / (35)) + (100026331.88254677 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.08128866880067842 = (562555.675765913 - 608285.0777751901) / 562555.675765913 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Xml.Linq.Perf_XDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetRootElement - Duration of single invocation 28.57 ns 34.09 ns 1.19 0.41 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XDocument*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Xml.Linq.Perf_XDocument.GetRootElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.09038624759987 > 30.332454428089143.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.55694015820251 (T) = (0 -35.26728468953505) / Math.Sqrt((10.290388167961012 / (35)) + (4.6589592388838295 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.19422522016267782 = (29.531518924655543 - 35.26728468953505) / 29.531518924655543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a7d80c177a7ea412fb00867ccc4f4b2c44c9e030
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 700.57 μs 773.98 μs 1.10 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 773.9783272916667 > 738.8892487713067.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -20.656750701736485 (T) = (0 -760488.7385758074) / Math.Sqrt((93826734.68115443 / (34)) + (123365892.39736505 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.07666274390570096 = (706338.8631959708 - 760488.7385758074) / 706338.8631959708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AndNotBenchmark - Duration of single invocation 94.85 ns 108.93 ns 1.15 0.23 False
AddOperatorBenchmark - Duration of single invocation 89.34 ns 95.22 ns 1.07 0.18 False
LessThanAnyBenchmark - Duration of single invocation 120.40 ns 136.16 ns 1.13 0.25 False
GreaterThanAnyBenchmark - Duration of single invocation 122.10 ns 131.84 ns 1.08 0.27 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 123.06 ns 145.54 ns 1.18 0.25 False
AbsBenchmark - Duration of single invocation 85.54 ns 99.70 ns 1.17 0.18 False
GreaterThanOrEqualBenchmark - Duration of single invocation 94.21 ns 104.19 ns 1.11 0.20 False
DivisionOperatorBenchmark - Duration of single invocation 87.54 ns 96.73 ns 1.11 0.15 False
OneBenchmark - Duration of single invocation 47.59 ns 57.53 ns 1.21 0.12 False
SubtractBenchmark - Duration of single invocation 87.50 ns 98.48 ns 1.13 0.23 False
OnesComplementBenchmark - Duration of single invocation 72.10 ns 83.51 ns 1.16 0.20 False
MinBenchmark - Duration of single invocation 108.02 ns 116.96 ns 1.08 0.15 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 129.32 ns 139.46 ns 1.08 0.18 False
MultiplyBenchmark - Duration of single invocation 87.58 ns 98.60 ns 1.13 0.24 False
LessThanBenchmark - Duration of single invocation 100.13 ns 111.01 ns 1.11 0.17 False
SubtractionOperatorBenchmark - Duration of single invocation 85.45 ns 96.41 ns 1.13 0.18 False
GreaterThanBenchmark - Duration of single invocation 88.83 ns 97.29 ns 1.10 0.17 False
OnesComplementOperatorBenchmark - Duration of single invocation 71.80 ns 84.98 ns 1.18 0.24 False
LessThanOrEqualAllBenchmark - Duration of single invocation 159.89 ns 175.30 ns 1.10 0.24 False
LessThanAllBenchmark - Duration of single invocation 160.61 ns 174.31 ns 1.09 0.21 False
DotBenchmark - Duration of single invocation 60.12 ns 64.76 ns 1.08 0.15 False
EqualsAnyBenchmark - Duration of single invocation 121.25 ns 129.74 ns 1.07 0.31 False
NegateBenchmark - Duration of single invocation 87.21 ns 100.57 ns 1.15 0.20 False
MultiplyOperatorBenchmark - Duration of single invocation 86.27 ns 96.50 ns 1.12 0.20 False
DivideBenchmark - Duration of single invocation 88.42 ns 99.57 ns 1.13 0.17 False
GreaterThanAllBenchmark - Duration of single invocation 134.60 ns 145.63 ns 1.08 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Single>.AndNotBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.9341872531174 > 104.03534127712646.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.771667072412189 (T) = (0 -109.0319273494989) / Math.Sqrt((34.63753918373406 / (35)) + (8.64639968366473 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.11321892794500275 = (97.94293342709449 - 109.0319273494989) / 97.94293342709449 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.21675163767978 > 90.82276111637388.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.785796255737884 (T) = (0 -97.49619856563888) / Math.Sqrt((4.97427778542565 / (35)) + (28.751378889370812 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.13585894364453477 = (85.83477650209898 - 97.49619856563888) / 85.83477650209898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 136.16244037099844 > 123.13811696342678.
IsChangePoint: Marked as a change because one of 9/18/2022 2:31:46 PM, 9/29/2022 2:35:34 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.670536739403931 (T) = (0 -134.25977040539325) / Math.Sqrt((168.79687849914495 / (39)) + (62.542680327817884 / (24))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (24) - 2, .025) and -0.10084162853355712 = (121.96102229913137 - 134.25977040539325) / 121.96102229913137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.84329563545364 > 128.68036272570836.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.603943818025904 (T) = (0 -131.32625085352404) / Math.Sqrt((6.468463089335138 / (35)) + (21.448794976695464 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07563058941104307 = (122.0923355530742 - 131.32625085352404) / 122.0923355530742 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.53518123393317 > 124.7038150476969.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.869619187729905 (T) = (0 -135.52601216753857) / Math.Sqrt((28.49693177993899 / (35)) + (65.33869410730213 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.12765922397266602 = (120.18348210737791 - 135.52601216753857) / 120.18348210737791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.AbsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.70367157656436 > 90.31338567482587.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.64195833589297 (T) = (0 -102.02559505596955) / Math.Sqrt((6.5971296434630275 / (35)) + (42.292025095226776 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.17471762198229251 = (86.85116588598129 - 102.02559505596955) / 86.85116588598129 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 104.18978702959848 > 98.95445821519701.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.073290050300043 (T) = (0 -104.75142198113124) / Math.Sqrt((10.145881887194792 / (35)) + (15.440826161189822 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.10475277422706386 = (94.81888113331074 - 104.75142198113124) / 94.81888113331074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.73286917637509 > 91.92649501383501.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.34907362608478 (T) = (0 -98.56588316110567) / Math.Sqrt((10.9483523291834 / (34)) + (27.310316419973205 / (31))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (31) - 2, .025) and -0.13014914646091055 = (87.21493394899879 - 98.56588316110567) / 87.21493394899879 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OneBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.52761371632606 > 49.50634647056762.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -30.751881691688258 (T) = (0 -58.14879120574223) / Math.Sqrt((1.7457087169255048 / (35)) + (2.222040779617673 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.23166996863920444 = (47.2113412572584 - 58.14879120574223) / 47.2113412572584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.SubtractBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.48485833037445 > 91.66633480395278.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.755100423142443 (T) = (0 -100.70659342143416) / Math.Sqrt((8.693156028739757 / (34)) + (36.677679561774625 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.14922380713180963 = (87.6300967631135 - 100.70659342143416) / 87.6300967631135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.50896400989305 > 76.36619588660815.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -22.02277112172153 (T) = (0 -84.71872516354621) / Math.Sqrt((5.193318826111116 / (35)) + (3.0941616728735113 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.15113033841520337 = (73.59611882020336 - 84.71872516354621) / 73.59611882020336 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 116.96078674416825 > 113.34120692336364.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.936530818402666 (T) = (0 -118.64061156367025) / Math.Sqrt((4.4312100293777625 / (35)) + (22.458943325480487 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.0974966590841435 = (108.1011140960332 - 118.64061156367025) / 108.1011140960332 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 139.46478231590564 > 135.4335572263413.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.247102930620327 (T) = (0 -140.23378610102105) / Math.Sqrt((10.022738476122479 / (36)) + (42.841048238665124 / (27))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (27) - 2, .025) and -0.08732988211089072 = (128.97078283986625 - 140.23378610102105) / 128.97078283986625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.MultiplyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.60062636958752 > 92.1936330817685.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -3.739455652547932 (T) = (0 -101.20426438680643) / Math.Sqrt((165.7019930777288 / (41)) + (35.598419449487054 / (24))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (24) - 2, .025) and -0.09510966726401404 = (92.41473015177725 - 101.20426438680643) / 92.41473015177725 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 111.01410851372984 > 106.0217776877353.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.403234580672489 (T) = (0 -112.62839923311998) / Math.Sqrt((17.559217032270197 / (35)) + (22.78826959736417 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.11707840465871332 = (100.8240771313898 - 112.62839923311998) / 100.8240771313898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.SubtractionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.40631630226079 > 90.21224747816916.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.37364721328412 (T) = (0 -98.00943915505728) / Math.Sqrt((11.0451309440451 / (35)) + (30.616335562794408 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.14148177993061753 = (85.86158875090811 - 98.00943915505728) / 85.86158875090811 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.29480088495578 > 92.83231892785412.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.563857110487485 (T) = (0 -97.98833669182595) / Math.Sqrt((5.844765815117476 / (34)) + (28.53479087887986 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.11536618477630073 = (87.8530638899355 - 97.98833669182595) / 87.8530638899355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.98052992064308 > 78.0211707384833.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -13.145678006949632 (T) = (0 -84.71100867074473) / Math.Sqrt((18.12354846143993 / (35)) + (4.1664367675047655 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.1444434192618992 = (74.01939427060405 - 84.71100867074473) / 74.01939427060405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 175.29889965399335 > 169.65226269610386.
IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.848392499113156 (T) = (0 -179.45715003214306) / Math.Sqrt((27.408301794729102 / (36)) + (100.23214919483338 / (29))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (29) - 2, .025) and -0.09867897394445682 = (163.33902285201594 - 179.45715003214306) / 163.33902285201594 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 174.30616715566617 > 168.65066955296155.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.094496988449237 (T) = (0 -178.90226969176987) / Math.Sqrt((56.092365919763 / (34)) + (149.29430895841023 / (31))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (31) - 2, .025) and -0.11214377496447489 = (160.8625374875515 - 178.90226969176987) / 160.8625374875515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.DotBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.75938784854365 > 63.51661316274169.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.400237653539252 (T) = (0 -65.96994699688406) / Math.Sqrt((3.6138929772492645 / (35)) + (4.986844016798541 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.08079238163427033 = (61.0385011200122 - 65.96994699688406) / 61.0385011200122 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.73800735117445 > 127.6886305281533.
IsChangePoint: Marked as a change because one of 7/31/2022 3:07:22 PM, 9/29/2022 2:35:34 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.936017366045757 (T) = (0 -131.47729813456775) / Math.Sqrt((22.271630633999695 / (39)) + (29.30336347983258 / (26))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (26) - 2, .025) and -0.07382018683924937 = (122.43884008324186 - 131.47729813456775) / 122.43884008324186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 100.57371220075903 > 91.42261391381436.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.927155679877984 (T) = (0 -100.26429527359993) / Math.Sqrt((10.225065119907073 / (35)) + (25.849881570148394 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.1486504624088845 = (87.28877805292599 - 100.26429527359993) / 87.28877805292599 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.50370756358815 > 90.62129783652658.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.809632513426406 (T) = (0 -100.18086195669318) / Math.Sqrt((4.361649618994744 / (35)) + (47.31099917172178 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.16684158965555007 = (85.85643745031956 - 100.18086195669318) / 85.85643745031956 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.DivideBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.57283877959979 > 92.78951681913854.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.742790495665291 (T) = (0 -102.11107692969759) / Math.Sqrt((3.2252708808401174 / (35)) + (46.293749953767396 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.15835625944574175 = (88.15170298173759 - 102.11107692969759) / 88.15170298173759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.63370965549228 > 141.87989857100644.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -9.528288213438907 (T) = (0 -146.94864421739825) / Math.Sqrt((10.877222457472886 / (38)) + (32.839838710065266 / (27))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (27) - 2, .025) and -0.08634324840352252 = (135.26907304238534 - 146.94864421739825) / 135.26907304238534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanAnyBenchmark - Duration of single invocation 60.08 ns 66.82 ns 1.11 0.30 False
EqualsAnyBenchmark - Duration of single invocation 75.69 ns 82.10 ns 1.08 0.21 False
MultiplyOperatorBenchmark - Duration of single invocation 47.06 ns 50.89 ns 1.08 0.20 False
NegateBenchmark - Duration of single invocation 48.01 ns 57.46 ns 1.20 0.29 False
LessThanOrEqualBenchmark - Duration of single invocation 47.36 ns 53.59 ns 1.13 0.22 False
LessThanOrEqualAllBenchmark - Duration of single invocation 106.50 ns 113.20 ns 1.06 0.32 False
DotBenchmark - Duration of single invocation 32.36 ns 35.97 ns 1.11 0.19 False
AddOperatorBenchmark - Duration of single invocation 47.34 ns 49.99 ns 1.06 0.20 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt64>.LessThanAnyBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.81992843005244 > 64.7977591254146.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.241344489633605 (T) = (0 -68.42089681367001) / Math.Sqrt((3.095237229611062 / (35)) + (7.066879586415793 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.1031672937440239 = (62.02223108106958 - 68.42089681367001) / 62.02223108106958 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.10225564157454 > 81.52411099322205.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.261784346251554 (T) = (0 -83.27734173572067) / Math.Sqrt((5.924569973549563 / (35)) + (5.146811860860606 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.061479444140557504 = (78.45403148918008 - 83.27734173572067) / 78.45403148918008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.88645378478134 > 49.51801565465606.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.374174435717273 (T) = (0 -50.9149599731942) / Math.Sqrt((2.7999300167478367 / (36)) + (4.390723628749405 / (28))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (28) - 2, .025) and -0.06455010897173939 = (47.827678137550066 - 50.9149599731942) / 47.827678137550066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.456562383927945 > 50.53088338009733.
IsChangePoint: Marked as a change because one of 10/1/2022 3:26:37 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.827872898401252 (T) = (0 -54.58809407922634) / Math.Sqrt((11.44630900096086 / (47)) + (8.353249349638864 / (17))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (17) - 2, .025) and -0.08203825980482624 = (50.44931968401258 - 54.58809407922634) / 50.44931968401258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.58700011330024 > 49.6834674627036.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.7706919600671664 (T) = (0 -52.67641842718309) / Math.Sqrt((1.3976532672667243 / (35)) + (14.484951511816492 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.12150865261102553 = (46.96924834645295 - 52.67641842718309) / 46.96924834645295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.20316582415087 > 111.70077330734254.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.487602193323859 (T) = (0 -114.23582555336357) / Math.Sqrt((28.29793878108734 / (41)) + (26.037645190230563 / (24))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (24) - 2, .025) and -0.05522932585859021 = (108.25687151976683 - 114.23582555336357) / 108.25687151976683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.DotBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.97018916622742 > 34.0032401228205.
IsChangePoint: Marked as a change because one of 9/28/2022 6:26:25 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.279760281522143 (T) = (0 -35.937255895979426) / Math.Sqrt((2.569592864388539 / (31)) + (5.516382990633066 / (33))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (33) - 2, .025) and -0.09574687642997812 = (32.79704160605557 - 35.937255895979426) / 32.79704160605557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.98619002200746 > 49.795285932894615.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.49183769975758 (T) = (0 -50.6913194965141) / Math.Sqrt((7.093196651819439 / (34)) + (2.7883475092204155 / (31))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (31) - 2, .025) and -0.05088242036967194 = (48.23690882437853 - 50.6913194965141) / 48.23690882437853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqOrderByExtension - Duration of single invocation 658.49 μs 726.22 μs 1.10 0.03 False
LinqQuery - Duration of single invocation 665.68 μs 722.87 μs 1.09 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 726.2216455692308 > 696.3148888574999.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -19.64750531554535 (T) = (0 -709619.2257979033) / Math.Sqrt((34906037.81707124 / (35)) + (144573749.04466844 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07264104000447087 = (661562.628439935 - 709619.2257979033) / 661562.628439935 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqQuery(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 722.8721083571429 > 695.9999262221924.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -17.81841543109408 (T) = (0 -710106.0596031818) / Math.Sqrt((76112693.02492712 / (35)) + (148025041.1960429 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07261465142518782 = (662032.7800479517 - 710106.0596031818) / 662032.7800479517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.Tests.Perf_Dictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ContainsValue - Duration of single invocation 446.98 ms 474.59 ms 1.06 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_Dictionary*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Tests.Perf_Dictionary.ContainsValue(Items: 3000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 474.59349592857143 > 461.8754490275.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.274807398500947 (T) = (0 -474873378.7569408) / Math.Sqrt((77751045621464.12 / (35)) + (42632425418503.086 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07048948738324498 = (443603962.8168079 - 474873378.7569408) / 443603962.8168079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Convert_int_float - Duration of single invocation 85.19 μs 94.65 μs 1.11 0.28 False
Convert_uint_float - Duration of single invocation 120.27 μs 131.45 μs 1.09 0.17 False
Narrow_long - Duration of single invocation 88.77 μs 95.35 μs 1.07 0.24 False
Convert_float_int - Duration of single invocation 84.54 μs 90.24 μs 1.07 0.19 False
Convert_float_uint - Duration of single invocation 97.26 μs 113.24 μs 1.16 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorConvert.Convert_int_float

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 94.6464361185984 > 89.72459058218904.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -6.6297351645085545 (T) = (0 -97662.0983694668) / Math.Sqrt((27653522.722300697 / (35)) + (35720981.71417104 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.10563349092522303 = (88331.34955756504 - 97662.0983694668) / 88331.34955756504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_uint_float

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.44584385408407 > 126.01146452412965.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.253017207386935 (T) = (0 -132369.04909733855) / Math.Sqrt((14649406.3253214 / (34)) + (60775480.038222015 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.09395242982388935 = (121000.73594484183 - 132369.04909733855) / 121000.73594484183 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Narrow_long

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.35219389329806 > 93.75595800366624.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.529464442975978 (T) = (0 -95407.66087306321) / Math.Sqrt((8184774.289811758 / (35)) + (8840254.273463301 / (32))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (32) - 2, .025) and -0.05973217058802504 = (90029.97504560358 - 95407.66087306321) / 90029.97504560358 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_float_int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.24385291116197 > 89.41116009014492.
IsChangePoint: Marked as a change because one of 9/29/2022 2:35:34 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.731361975406741 (T) = (0 -92738.18677361902) / Math.Sqrt((5391835.352982037 / (39)) + (32596354.169761814 / (26))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (26) - 2, .025) and -0.07864885584040063 = (85976.25285697311 - 92738.18677361902) / 85976.25285697311 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorConvert.Convert_float_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.23577076761302 > 102.24179881607672.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -8.049876755838342 (T) = (0 -107212.92173902133) / Math.Sqrt((4574102.338061692 / (34)) + (39201611.443324134 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.09906883994006377 = (97548.86849932716 - 107212.92173902133) / 97548.86849932716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 579.09 μs 616.45 μs 1.06 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.CreateAddAndClear<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 616.44937 > 614.341326803775.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.73910885552949 (T) = (0 -622573.4931288407) / Math.Sqrt((92885576.9762894 / (35)) + (73299810.53754686 / (31))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (31) - 2, .025) and -0.06409184158486998 = (585074.9613882698 - 622573.4931288407) / 585074.9613882698 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HeapSort - Duration of single invocation 139.47 μs 147.76 μs 1.06 0.06 False
Dequeue_And_Enqueue - Duration of single invocation 353.08 μs 373.93 μs 1.06 0.06 False
HeapSort - Duration of single invocation 2.17 ms 2.30 ms 1.06 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 147.75886416666668 > 146.31118397331304.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -17.573352863241933 (T) = (0 -150464.5955435361) / Math.Sqrt((2991227.8203371526 / (35)) + (8584950.926842248 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.0789011316089023 = (139460.96740037433 - 150464.5955435361) / 139460.96740037433 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.Dequeue_And_Enqueue(Size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 373.93248316375974 > 371.29877394743204.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.838799409535437 (T) = (0 -381353.24727211567) / Math.Sqrt((10067967.837499997 / (35)) + (64779817.65787117 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07540129608584924 = (354614.8295153926 - 381353.24727211567) / 354614.8295153926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;.HeapSort(Size: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.297940103954082 > 2.283584146484375.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -18.921994895739786 (T) = (0 -2345896.1522303983) / Math.Sqrt((954908414.8371632 / (35)) + (1593479301.6858375 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.07796280151378329 = (2176231.0804566317 - 2345896.1522303983) / 2176231.0804566317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 623.40 μs 685.09 μs 1.10 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.TryGetValueFalse<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 685.0936748835405 > 651.4475297500001.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -12.358887607644494 (T) = (0 -675132.7814385307) / Math.Sqrt((357590210.5888331 / (35)) + (233967478.01165384 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.08424500090034068 = (622675.4846717399 - 675132.7814385307) / 622675.4846717399 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Regressions in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateViewBetween - Duration of single invocation 43.76 μs 46.34 μs 1.06 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_SortedSet*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.34042077383695 > 46.17975092120243.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -15.756386543608366 (T) = (0 -46767.949806973156) / Math.Sqrt((353423.92478436034 / (35)) + (417461.76158661366 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.05566144472780908 = (44302.03455903589 - 46767.949806973156) / 44302.03455903589 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 278.80 ns 199.60 ns 0.72 0.08 False
InequalityOperatorBenchmark - Duration of single invocation 20.88 ns 19.78 ns 0.95 0.18 False
LessThanOrEqualAnyBenchmark - Duration of single invocation 301.66 ns 277.45 ns 0.92 0.06 False
SubtractBenchmark - Duration of single invocation 276.30 ns 235.90 ns 0.85 0.10 False
DivisionOperatorBenchmark - Duration of single invocation 302.60 ns 252.62 ns 0.83 0.07 False
EqualityOperatorBenchmark - Duration of single invocation 21.15 ns 19.05 ns 0.90 0.22 False
AddOperatorBenchmark - Duration of single invocation 271.40 ns 240.15 ns 0.88 0.06 False
LessThanOrEqualBenchmark - Duration of single invocation 284.05 ns 248.46 ns 0.87 0.06 False
LessThanBenchmark - Duration of single invocation 282.00 ns 240.50 ns 0.85 0.05 False
NegateBenchmark - Duration of single invocation 280.85 ns 234.07 ns 0.83 0.10 False
MaxBenchmark - Duration of single invocation 306.07 ns 258.03 ns 0.84 0.12 False
DivideBenchmark - Duration of single invocation 308.77 ns 248.27 ns 0.80 0.08 False
GreaterThanBenchmark - Duration of single invocation 275.01 ns 227.97 ns 0.83 0.13 False
EqualsStaticBenchmark - Duration of single invocation 277.00 ns 241.41 ns 0.87 0.12 False
EqualsAllBenchmark - Duration of single invocation 23.66 ns 18.98 ns 0.80 0.19 False
MultiplyBenchmark - Duration of single invocation 271.49 ns 238.00 ns 0.88 0.10 False
MinBenchmark - Duration of single invocation 325.11 ns 292.29 ns 0.90 0.07 False
MultiplyOperatorBenchmark - Duration of single invocation 273.73 ns 242.30 ns 0.89 0.08 False
GreaterThanOrEqualBenchmark - Duration of single invocation 294.32 ns 260.25 ns 0.88 0.09 False
SquareRootBenchmark - Duration of single invocation 235.05 ns 194.71 ns 0.83 0.09 False
AddBenchmark - Duration of single invocation 269.12 ns 233.10 ns 0.87 0.05 False
UnaryNegateOperatorBenchmark - Duration of single invocation 273.73 ns 232.03 ns 0.85 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.DotBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 199.60468653779978 < 261.61840685129033.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.29518352899801 (T) = (0 -203.42286657980128) / Math.Sqrt((107.66293732380517 / (299)) + (140.72271295661173 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.24451429058669075 = (269.26103835607194 - 203.42286657980128) / 269.26103835607194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.77506556296799 < 20.10972625347177.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 18.890862435368234 (T) = (0 -19.627811547953833) / Math.Sqrt((0.9698744953774194 / (299)) + (0.14443201476456421 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07952589398956474 = (21.32358902851235 - 19.627811547953833) / 21.32358902851235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 277.4473283882125 < 286.74947250537264.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 10:18:57 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.413288899653555 (T) = (0 -284.3230477438122) / Math.Sqrt((75.73190446694427 / (299)) + (17.54546824919007 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05019653537097506 = (299.3493478725763 - 284.3230477438122) / 299.3493478725763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.SubtractBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 235.89540580576008 < 263.50930920858616.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 9.944345700373564 (T) = (0 -246.8571849938849) / Math.Sqrt((200.0508341847458 / (299)) + (127.26365512265633 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08306806986211343 = (269.22084058820263 - 246.8571849938849) / 269.22084058820263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 252.6192692441687 < 287.56530181337655.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 22.81603364443052 (T) = (0 -256.4708082279755) / Math.Sqrt((223.2227655384743 / (299)) + (55.16127742238876 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.12513946565473327 = (293.15622108832997 - 256.4708082279755) / 293.15622108832997 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.052782403819982 < 20.042070168516698.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 12.154412065065099 (T) = (0 -19.231175331655404) / Math.Sqrt((1.529199914925983 / (299)) + (0.5391146339576827 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08867797517664817 = (21.102502526901095 - 19.231175331655404) / 21.102502526901095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.AddOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 240.15234606725988 < 257.7979407609941.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.31304954360344 (T) = (0 -241.40295699577672) / Math.Sqrt((145.73964616969093 / (299)) + (16.856499876225804 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08002621535409611 = (262.40199560544244 - 241.40295699577672) / 262.40199560544244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 248.46304342361063 < 268.2010481585252.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.76493660169012 (T) = (0 -250.94116040562014) / Math.Sqrt((86.97051149421483 / (299)) + (24.605165251327655 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1056253425068097 = (280.57722600165556 - 250.94116040562014) / 280.57722600165556 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.LessThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 240.49998957660648 < 267.29067409653607.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 17.318710750351535 (T) = (0 -247.0459160686878) / Math.Sqrt((94.2585404941513 / (299)) + (90.81288338068889 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.11211906572018539 = (278.2421679873932 - 247.0459160686878) / 278.2421679873932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 234.0662840454773 < 266.24898486114876.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 9.320645197592246 (T) = (0 -242.99111951898215) / Math.Sqrt((241.20444674530683 / (299)) + (198.28503757407816 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09457945061908572 = (268.3737625406542 - 242.99111951898215) / 268.3737625406542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 258.02514838224874 < 290.6441587171346.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 13.056768508599388 (T) = (0 -276.91821117179376) / Math.Sqrt((304.374647962117 / (299)) + (126.62447371819137 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09861860939231756 = (307.21536306080645 - 276.91821117179376) / 307.21536306080645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 248.26963458558615 < 293.77882416967026.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.913836626222462 (T) = (0 -253.77496103574964) / Math.Sqrt((269.82732577270895 / (299)) + (33.35278573398818 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14570613765735357 = (297.0581578800618 - 253.77496103574964) / 297.0581578800618 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 227.97242360249535 < 261.2300509368548.
IsChangePoint: Marked as a change because one of 7/28/2022 1:17:22 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 6.621104494353934 (T) = (0 -247.78620484120856) / Math.Sqrt((176.46306360158613 / (299)) + (393.8721905076889 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0913990585047188 = (272.71180726868687 - 247.78620484120856) / 272.71180726868687 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 241.4135140853529 < 262.50262021433525.
IsChangePoint: Marked as a change because one of 7/30/2022 2:52:48 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.658343214966926 (T) = (0 -243.82407403549232) / Math.Sqrt((156.93958852155416 / (299)) + (39.95902974925473 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10464281611392774 = (272.3204531371886 - 243.82407403549232) / 272.3204531371886 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.982082530168864 < 20.183973243107186.
IsChangePoint: Marked as a change because one of 8/3/2022 7:52:40 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 14.299974394918454 (T) = (0 -19.099320678500884) / Math.Sqrt((2.2963951482271723 / (299)) + (0.36549561481334886 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09635758626081452 = (21.135927650263483 - 19.099320678500884) / 21.135927650263483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 237.99767825744414 < 259.9704810360291.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 14.560887042443309 (T) = (0 -244.7387371919347) / Math.Sqrt((173.9175059358691 / (299)) + (50.08155775705662 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08194913116811987 = (266.5851593859261 - 244.7387371919347) / 266.5851593859261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MinBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 292.29306696304616 < 309.0920934866548.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.346120599982218 (T) = (0 -292.7916123092085) / Math.Sqrt((139.80900278061839 / (299)) + (49.46848564625857 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09181437037713978 = (322.3918136986987 - 292.7916123092085) / 322.3918136986987 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 242.30427481629926 < 259.9202651436803.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 18.55267068957184 (T) = (0 -245.18800592733098) / Math.Sqrt((118.15144286735239 / (299)) + (20.470181546044376 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07282437930697765 = (264.4461312993367 - 245.18800592733098) / 264.4461312993367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 260.2520380018609 < 280.224586049106.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 15.698921499757601 (T) = (0 -263.72873710921556) / Math.Sqrt((109.64671301562447 / (299)) + (80.16844103784354 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09529510141153856 = (291.50802379946253 - 263.72873710921556) / 291.50802379946253 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.SquareRootBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 194.71052325439004 < 222.69967358626917.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 9.431093191614783 (T) = (0 -205.26378452801592) / Math.Sqrt((31.1133740650836 / (299)) + (256.76090636679993 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.12274683739316067 = (233.98466175722353 - 205.26378452801592) / 233.98466175722353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.AddBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 233.0970542989224 < 256.15514958786576.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.73833945560925 (T) = (0 -239.82930869693863) / Math.Sqrt((109.07559747151609 / (299)) + (47.49621861672778 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09107270771005678 = (263.85972863980663 - 239.82930869693863) / 263.85972863980663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 232.0342716521914 < 260.1964711894533.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.69606819228725 (T) = (0 -241.15574457466283) / Math.Sqrt((106.57367155922077 / (299)) + (133.13253303320022 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09737143213142366 = (267.1705208091479 - 241.15574457466283) / 267.1705208091479 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 1.48 μs 1.39 μs 0.93 0.02 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 111.44 μs 102.06 μs 0.92 0.03 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 156.31 μs 146.64 μs 0.94 0.03 False
ReadSpanEmptyLoop - Duration of single invocation 38.57 μs 34.68 μs 0.90 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 72.97 μs 67.33 μs 0.92 0.02 False
ReadReturnBytes - Duration of single invocation 189.76 μs 176.29 μs 0.93 0.02 False
ReadSpanEmptyLoop - Duration of single invocation 1.34 μs 1.21 μs 0.90 0.06 False
ReadReturnBytes - Duration of single invocation 63.49 μs 59.42 μs 0.94 0.02 False
ReadReturnBytes - Duration of single invocation 135.21 μs 126.71 μs 0.94 0.01 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 107.34 μs 97.55 μs 0.91 0.02 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 37.91 μs 34.12 μs 0.90 0.03 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 23.05 μs 21.70 μs 0.94 0.02 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 47.77 μs 44.33 μs 0.93 0.03 False
ReadSpanEmptyLoop - Duration of single invocation 15.17 μs 13.58 μs 0.89 0.07 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 3.10 μs 2.83 μs 0.91 0.14 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 28.06 μs 26.36 μs 0.94 0.03 False
ReadSpanEmptyLoop - Duration of single invocation 13.72 μs 12.31 μs 0.90 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 1.04 ms 938.86 μs 0.90 0.02 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 3.73 μs 3.38 μs 0.91 0.08 False
ReadReturnBytes - Duration of single invocation 129.36 μs 116.00 μs 0.90 0.03 False
ReadSpanEmptyLoop - Duration of single invocation 694.39 μs 643.13 μs 0.93 0.02 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 129.35 μs 119.22 μs 0.92 0.02 False
ReadSpanEmptyLoop - Duration of single invocation 159.10 μs 142.31 μs 0.89 0.02 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 131.54 μs 120.14 μs 0.91 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 28.31 μs 25.31 μs 0.89 0.03 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 20.85 μs 19.55 μs 0.94 0.01 False
ReadReturnBytes - Duration of single invocation 1.32 ms 1.22 ms 0.93 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 21.47 μs 19.78 μs 0.92 0.02 False
ReadReturnBytes - Duration of single invocation 18.08 μs 16.60 μs 0.92 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 105.29 μs 96.07 μs 0.91 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 23.68 μs 21.02 μs 0.89 0.04 False
ReadReturnBytes - Duration of single invocation 47.63 μs 42.20 μs 0.89 0.04 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 1.02 ms 936.45 μs 0.91 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 1.22 ms 1.13 ms 0.93 0.02 False
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 14.19 μs 12.71 μs 0.90 0.01 False
ReadSpanEmptyLoop - Duration of single invocation 10.01 μs 9.30 μs 0.93 0.01 False
ReadMultiSpanSequenceEmptyLoop - Duration of single invocation 29.49 μs 27.22 μs 0.92 0.02 False
ReadSpanEmptyLoop - Duration of single invocation 111.51 μs 101.70 μs 0.91 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3868156625146673 < 1.4113750328656618.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.24970465838624 (T) = (0 -1395.6547987290837) / Math.Sqrt((520.4412145287215 / (299)) + (538.8245732310486 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.05885800370434373 = (1482.9375420737722 - 1395.6547987290837) / 1482.9375420737722 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.06465141612199 < 105.93721390115884.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 34.46533158814249 (T) = (0 -102735.2763825663) / Math.Sqrt((4196102.472657745 / (299)) + (2478534.9252405358 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09701065751767607 = (113772.41297239048 - 102735.2763825663) / 113772.41297239048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.63627924382718 < 148.84513522411615.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 42.5802833616096 (T) = (0 -145224.07296764324) / Math.Sqrt((5246476.388525372 / (299)) + (2697872.4197509983 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08769325909494251 = (159183.3825798252 - 145224.07296764324) / 159183.3825798252 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.67913363492947 < 36.09078504733425.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 30.248902108806853 (T) = (0 -34409.08117402018) / Math.Sqrt((432239.6310263444 / (299)) + (343461.993583041 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09076386199719479 = (37843.94365318773 - 34409.08117402018) / 37843.94365318773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.33048045977011 < 69.23664888392857.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 33.500988320729654 (T) = (0 -68396.23332141298) / Math.Sqrt((1754491.9081144088 / (299)) + (925613.2356121765 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.0847356830233543 = (74728.39490492035 - 68396.23332141298) / 74728.39490492035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 176.28971210205992 < 184.42175426504627.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.605752067317184 (T) = (0 -178463.66061187402) / Math.Sqrt((10334517.630608076 / (299)) + (5931227.159060574 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08888399159856279 = (195873.69661629634 - 178463.66061187402) / 195873.69661629634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2066159751908394 < 1.2442728089090567.
IsChangePoint: Marked as a change because one of 7/1/2022 7:34:46 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 22.298204471303173 (T) = (0 -1218.8326762341399) / Math.Sqrt((492.8660884153523 / (299)) + (331.7512741994832 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06207703373798522 = (1299.5018994914462 - 1218.8326762341399) / 1299.5018994914462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.418587605241726 < 60.191223800601286.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 28.65855184270399 (T) = (0 -59744.3198868602) / Math.Sqrt((1487352.9237825845 / (299)) + (761126.2611996373 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07711697640673629 = (64736.61163930016 - 59744.3198868602) / 64736.61163930016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 126.71091178333333 < 128.76133635442216.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.3839482720021 (T) = (0 -125964.69914976857) / Math.Sqrt((3961782.908362022 / (299)) + (2460433.3365034377 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08596637879208016 = (137811.88812650333 - 125964.69914976857) / 137811.88812650333 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.55049299948239 < 101.95173252237274.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 53.114053795017774 (T) = (0 -97774.74927991518) / Math.Sqrt((2259079.221116718 / (299)) + (921424.3886606154 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0972531214623212 = (108308.0447071679 - 97774.74927991518) / 108308.0447071679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.12488667357968 < 36.33985507781794.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 50.78022336072366 (T) = (0 -34462.184204642814) / Math.Sqrt((444770.1493107875 / (299)) + (123225.49365499512 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10039955788535646 = (38308.322885696194 - 34462.184204642814) / 38308.322885696194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.700872029761904 < 22.058374572435387.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.5416007376715 (T) = (0 -21476.71802143161) / Math.Sqrt((93362.51893954187 / (299)) + (226213.17711345336 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07931582600846315 = (23326.91125592111 - 21476.71802143161) / 23326.91125592111 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.329834384005764 < 45.359216179769156.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 40.09558312380052 (T) = (0 -44341.86262087779) / Math.Sqrt((814762.6976165802 / (299)) + (301444.05369941506 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09504489128918471 = (48998.96380942753 - 44341.86262087779) / 48998.96380942753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.579424127991826 < 14.03961061020679.
IsChangePoint: Marked as a change because one of 8/2/2022 2:18:01 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 19.981284242761532 (T) = (0 -13938.21601237808) / Math.Sqrt((125123.3661118318 / (299)) + (77020.72072402024 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07577731548940501 = (15081.014831137589 - 13938.21601237808) / 15081.014831137589 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.8301040060078906 < 2.980978863693052.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 25.0480785835273 (T) = (0 -2875.118673236006) / Math.Sqrt((10832.537909137274 / (299)) + (4972.590502396041 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11017427280940974 = (3231.1031085980158 - 2875.118673236006) / 3231.1031085980158 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.355797532894734 < 26.65213663516061.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.564771089951606 (T) = (0 -25955.740863151714) / Math.Sqrt((247246.25231846556 / (299)) + (105389.3620416457 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09542772310342018 = (28693.938036881984 - 25955.740863151714) / 28693.938036881984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.30537730738363 < 13.02655414004439.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.894589565073915 (T) = (0 -12424.047766820731) / Math.Sqrt((36094.530257356644 / (299)) + (44728.20900594515 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.10651773513782657 = (13905.197960182497 - 12424.047766820731) / 13905.197960182497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 938.8645173319327 < 0.9806496215959201.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.523909754688137 (T) = (0 -948699.3563743726) / Math.Sqrt((174631332.8562607 / (299)) + (339717165.716951 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09100431141357977 = (1043678.6095759107 - 948699.3563743726) / 1043678.6095759107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.37698644989339 < 3.5372281436120137.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.321123771723727 (T) = (0 -3391.838522902733) / Math.Sqrt((8224.523027384808 / (299)) + (6028.794228707692 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10050552847288018 = (3770.8275373212996 - 3391.838522902733) / 3770.8275373212996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 116.00392229938271 < 122.22652718087615.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 54.017536284288106 (T) = (0 -118124.44759575032) / Math.Sqrt((6055515.838945445 / (299)) + (1040153.2522559541 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09774260992952255 = (130921.00867860261 - 118124.44759575032) / 130921.00867860261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 643.1251852500002 < 659.9925946173007.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 33.60376857816509 (T) = (0 -649559.1304563321) / Math.Sqrt((148220291.2222017 / (299)) + (78424593.47973557 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08470454230578975 = (709671.5328323441 - 649559.1304563321) / 709671.5328323441 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 119.21691237373737 < 122.86514502515003.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 34.515736764537785 (T) = (0 -120364.29813241762) / Math.Sqrt((3510299.7595229717 / (299)) + (3123366.3622062574 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08892539750664383 = (132112.45029003578 - 120364.29813241762) / 132112.45029003578 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 142.3139190151515 < 148.27002992358848.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.504332757933035 (T) = (0 -143762.6874946954) / Math.Sqrt((4546741.555567637 / (299)) + (3566118.708679193 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.0918409128230711 = (158301.21564008237 - 143762.6874946954) / 158301.21564008237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 120.13959109187569 < 124.52758401254303.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 54.465282311871725 (T) = (0 -121365.61969967476) / Math.Sqrt((3818052.76692931 / (299)) + (1024711.4414495052 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08960755099913463 = (133311.32066491843 - 121365.61969967476) / 133311.32066491843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.306271411649945 < 26.414630260699088.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 34.057171405724915 (T) = (0 -25688.421091976375) / Math.Sqrt((273114.89646317787 / (299)) + (143600.77744049326 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09217515020647364 = (28296.670990906332 - 25688.421091976375) / 28296.670990906332 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.549180264022787 < 19.79526837448247.
IsChangePoint: Marked as a change because one of 7/31/2022 3:07:22 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.913356064844777 (T) = (0 -19432.99188831154) / Math.Sqrt((106363.31101044861 / (299)) + (89328.33954068478 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08660288240418781 = (21275.512604486717 - 19432.99188831154) / 21275.512604486717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2204703881410255 < 1.2546172415538197.
IsChangePoint: Marked as a change because one of 7/31/2022 12:39:41 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 39.22702930829624 (T) = (0 -1230614.974785819) / Math.Sqrt((454239356.98416054 / (299)) + (167861945.47547898 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07933257424359076 = (1336655.2789404497 - 1230614.974785819) / 1336655.2789404497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.77808266322487 < 19.8819501641988.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.300733851426163 (T) = (0 -19722.890379137694) / Math.Sqrt((119655.3962695192 / (299)) + (246310.87866782755 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.0702030577412748 = (21212.040481898686 - 19722.890379137694) / 21212.040481898686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.59747314765459 < 17.15351607539604.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.91047431998685 (T) = (0 -16673.212142191194) / Math.Sqrt((87166.23848890196 / (299)) + (51997.86332281416 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09761491448630781 = (18476.82592482099 - 16673.212142191194) / 18476.82592482099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.07215707304528 < 101.39101805943352.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 42.18717768772821 (T) = (0 -97183.12853799453) / Math.Sqrt((1918526.772224569 / (299)) + (1604111.155480476 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0960304833123307 = (107507.0859624709 - 97183.12853799453) / 107507.0859624709 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.023132373935347 < 22.23695106221447.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.61678230537911 (T) = (0 -21010.962045088458) / Math.Sqrt((146925.68026048678 / (299)) + (109706.82209301037 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08956858972450078 = (23078.02851258227 - 21010.962045088458) / 23078.02851258227 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.201839152632594 < 44.0347172084077.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.8146068311999 (T) = (0 -42652.65257951491) / Math.Sqrt((533586.2294029575 / (299)) + (243286.91819704138 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08407202359178927 = (46567.69274236632 - 42652.65257951491) / 46567.69274236632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 936.4496433823529 < 0.9783236756458333.
IsChangePoint: Marked as a change because one of 7/31/2022 3:07:22 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.685099436991734 (T) = (0 -946662.4264844804) / Math.Sqrt((229847795.22389358 / (299)) + (148131793.4867419 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09139631188216753 = (1041887.0612835464 - 946662.4264844804) / 1041887.0612835464 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1336329504464284 < 1.1614202321686125.
IsChangePoint: Marked as a change because one of 7/31/2022 5:42:25 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.722951144446895 (T) = (0 -1131237.5902043853) / Math.Sqrt((348123674.67643803 / (299)) + (203461159.16845265 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09040569672629387 = (1243672.6858699163 - 1131237.5902043853) / 1243672.6858699163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.706380273224045 < 13.587532598182795.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 47.93726969339807 (T) = (0 -12982.492137832789) / Math.Sqrt((48333.24586458068 / (299)) + (26720.819362297367 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10835142924990171 = (14560.09975646715 - 12982.492137832789) / 14560.09975646715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.297044769889663 < 9.48432831889865.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 39.63725923401545 (T) = (0 -9105.38048327011) / Math.Sqrt((40734.125607229355 / (299)) + (16485.638649088043 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10491573855279854 = (10172.6517552082 - 9105.38048327011) / 10172.6517552082 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.217059039792385 < 27.551526464227702.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.378536010499136 (T) = (0 -26813.18883806878) / Math.Sqrt((187949.31650194665 / (299)) + (161028.73564013746 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07820835339655767 = (29088.123044798973 - 26813.18883806878) / 29088.123044798973 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 101.70149988328664 < 105.97066482544751.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 58.19431679253811 (T) = (0 -101888.92324281813) / Math.Sqrt((2962247.7772557647 / (299)) + (722427.7260505966 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09631132758375253 = (112747.8149863177 - 101888.92324281813) / 112747.8149863177 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 796.03 ns 716.37 ns 0.90 0.03 False
ToString - Duration of single invocation 2.91 μs 2.58 μs 0.88 0.18 False
ToStringWithFormat - Duration of single invocation 3.04 μs 2.82 μs 0.93 0.11 False
Parse - Duration of single invocation 1.21 μs 1.07 μs 0.88 0.04 False
ToStringWithFormat - Duration of single invocation 2.65 μs 2.31 μs 0.87 0.28 False
IsNaN - Duration of single invocation 7.57 ms 7.14 ms 0.94 0.02 False
TryParse - Duration of single invocation 1.24 μs 1.10 μs 0.89 0.02 False
Parse - Duration of single invocation 1.18 μs 1.07 μs 0.91 0.02 False
TryParse - Duration of single invocation 1.19 μs 1.07 μs 0.90 0.02 False
IsNaN - Duration of single invocation 7.57 ms 7.12 ms 0.94 0.05 False
TryParse - Duration of single invocation 809.34 ns 717.92 ns 0.89 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Single.Parse(value: "12345")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 716.3693509893543 < 764.1531786105614.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 34.65451702181101 (T) = (0 -728.6529011823453) / Math.Sqrt((322.0714017929019 / (299)) + (92.3789533139088 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08839128360126564 = (799.3044472642308 - 728.6529011823453) / 799.3044472642308 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5773392741046837 < 2.703548196175761.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.343969398037483 (T) = (0 -2605.567011591747) / Math.Sqrt((5701.413602985479 / (299)) + (6862.968471387703 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08987458180646483 = (2862.8658858505605 - 2605.567011591747) / 2862.8658858505605 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.8156366306306304 < 2.9149770270190625.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.291755633883895 (T) = (0 -2783.8159526629156) / Math.Sqrt((5195.872743899764 / (299)) + (8213.279730074048 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09211847343916812 = (3066.276679522665 - 2783.8159526629156) / 3066.276679522665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0718798249294823 < 1.1605092426427923.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 21.234048387938063 (T) = (0 -1093.058634029476) / Math.Sqrt((1950.8270736045806 / (299)) + (321.4735038515224 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07537577726043199 = (1182.1652592994524 - 1093.058634029476) / 1182.1652592994524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3093755579512787 < 2.452651034946843.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.510417527943773 (T) = (0 -2305.4143819012866) / Math.Sqrt((7348.171765140758 / (299)) + (4998.361690969696 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1109917458579556 = (2593.242943650927 - 2305.4143819012866) / 2593.242943650927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.IsNaN(value: NaN)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.136546612179487 < 7.187897325744047.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.590839928162357 (T) = (0 -7118663.316810844) / Math.Sqrt((58719886368.64804 / (299)) + (312173837.8360147 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06008924343856222 = (7573765.133674718 - 7118663.316810844) / 7573765.133674718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1028678426550833 < 1.1756941595406025.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 19.590310830682398 (T) = (0 -1089.3541844980944) / Math.Sqrt((2284.3486095621192 / (299)) + (355.8976216695409 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07357583671130793 = (1175.869788014834 - 1089.3541844980944) / 1175.869788014834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0690164524038681 < 1.1210240594194907.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.994637813007795 (T) = (0 -1068.732480149328) / Math.Sqrt((688.9924786591298 / (299)) + (172.164043490882 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0782582005633297 = (1159.4705597624977 - 1068.732480149328) / 1159.4705597624977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0680108367114713 < 1.1295944289657598.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 25.468082413006034 (T) = (0 -1064.2158420271603) / Math.Sqrt((987.2729818631748 / (299)) + (276.48474157240577 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07992585102190579 = (1156.663126780772 - 1064.2158420271603) / 1156.663126780772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.IsNaN(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.115812527777777 < 7.187183153159722.
IsChangePoint: Marked as a change because one of 9/18/2022 2:31:46 PM, 9/19/2022 9:42:49 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.00300811035977 (T) = (0 -7122345.052580638) / Math.Sqrt((36307837432.111984 / (299)) + (54001657.57258355 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.060068763154361035 = (7577517.134639405 - 7122345.052580638) / 7577517.134639405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "12345")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 717.9207829426597 < 769.8121299626944.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 18.727821669855427 (T) = (0 -730.7997611867412) / Math.Sqrt((614.2593473405608 / (299)) + (238.93430141038883 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07597229510673034 = (790.8851188300167 - 730.7997611867412) / 790.8851188300167 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsBenchmark - Duration of single invocation 42.52 ns 39.59 ns 0.93 0.21 False
DotBenchmark - Duration of single invocation 153.32 ns 124.18 ns 0.81 0.09 False
EqualityOperatorBenchmark - Duration of single invocation 22.65 ns 20.78 ns 0.92 0.17 False
EqualsAllBenchmark - Duration of single invocation 22.98 ns 20.09 ns 0.87 0.24 False
InequalityOperatorBenchmark - Duration of single invocation 25.47 ns 21.56 ns 0.85 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.59393077100062 < 40.76745065863077.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 14.474224653413271 (T) = (0 -39.840735381541215) / Math.Sqrt((3.3269836076430774 / (299)) + (0.9745244188028029 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07223708101742093 = (42.9427977410782 - 39.840735381541215) / 42.9427977410782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.18406187257786 < 145.64187105787818.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 15.6661125747731 (T) = (0 -126.18718802136804) / Math.Sqrt((20.224238575917667 / (299)) + (95.06834029791354 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1850644535836116 = (154.84315118694448 - 126.18718802136804) / 154.84315118694448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.781842632572015 < 21.592454870862728.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 6.503769145599808 (T) = (0 -21.280133488034405) / Math.Sqrt((1.5252805884384018 / (299)) + (1.922874053826859 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07550352989345836 = (23.018079761388403 - 21.280133488034405) / 23.018079761388403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.089508557039814 < 21.679693116920966.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 6.503963284607373 (T) = (0 -21.151005386125824) / Math.Sqrt((1.8817959955103478 / (299)) + (2.32168350509252 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08168763647934577 = (23.032473727171055 - 21.151005386125824) / 23.032473727171055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.557187219254207 < 23.05057180894635.
IsChangePoint: Marked as a change because one of 7/6/2022 10:15:52 AM, 7/6/2022 9:44:27 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 17.656091550082774 (T) = (0 -21.546899743898365) / Math.Sqrt((2.4376532529955623 / (299)) + (0.2083931673872542 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09213501793270358 = (23.73359493923203 - 21.546899743898365) / 23.73359493923203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MaxBenchmark - Duration of single invocation 308.04 ns 268.44 ns 0.87 0.09 False
EqualityOperatorBenchmark - Duration of single invocation 21.09 ns 19.54 ns 0.93 0.20 False
NegateBenchmark - Duration of single invocation 265.93 ns 230.83 ns 0.87 0.08 False
GreaterThanAllBenchmark - Duration of single invocation 326.73 ns 297.55 ns 0.91 0.07 False
DotBenchmark - Duration of single invocation 267.05 ns 197.62 ns 0.74 0.05 False
DivisionOperatorBenchmark - Duration of single invocation 309.60 ns 251.67 ns 0.81 0.11 False
GreaterThanBenchmark - Duration of single invocation 271.84 ns 235.18 ns 0.87 0.09 False
EqualsBenchmark - Duration of single invocation 41.47 ns 37.28 ns 0.90 0.21 False
GreaterThanOrEqualBenchmark - Duration of single invocation 294.31 ns 271.97 ns 0.92 0.10 False
AbsBenchmark - Duration of single invocation 231.48 ns 205.46 ns 0.89 0.09 False
LessThanOrEqualBenchmark - Duration of single invocation 281.93 ns 242.87 ns 0.86 0.10 False
DivideBenchmark - Duration of single invocation 311.17 ns 249.57 ns 0.80 0.12 False
EqualsStaticBenchmark - Duration of single invocation 274.65 ns 232.81 ns 0.85 0.08 False
SquareRootBenchmark - Duration of single invocation 236.26 ns 192.16 ns 0.81 0.05 False
UnaryNegateOperatorBenchmark - Duration of single invocation 268.78 ns 232.28 ns 0.86 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.MaxBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 268.43981145717396 < 292.50742499166864.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.680134405800136 (T) = (0 -271.4447381855212) / Math.Sqrt((77.84392651752574 / (299)) + (47.95340626938461 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.13139212329294772 = (312.50549927613537 - 271.4447381855212) / 312.50549927613537 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.54039876830688 < 20.260661885547485.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.70633913522231 (T) = (0 -19.687610830740088) / Math.Sqrt((2.7128351860657562 / (299)) + (0.4770787087173764 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08675415900578005 = (21.557843405349484 - 19.687610830740088) / 21.557843405349484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.NegateBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.82776393929004 < 255.52489256796582.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.12393795081912 (T) = (0 -238.190163055798) / Math.Sqrt((235.46759708393523 / (299)) + (99.53516824237961 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.12362328089768712 = (271.78969712908406 - 238.190163055798) / 271.78969712908406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 297.5492956279991 < 307.74497482446134.
IsChangePoint: Marked as a change because one of 9/28/2022 12:34:15 AM, 9/28/2022 11:19:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.28953794133528 (T) = (0 -299.48251867004285) / Math.Sqrt((103.08231767132145 / (299)) + (6.7187979787831 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.08694456758948696 = (328.00036891450685 - 299.48251867004285) / 328.00036891450685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.62258790841088 < 253.93263211824848.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 67.00150797728853 (T) = (0 -199.91988821724118) / Math.Sqrt((86.16248788557697 / (299)) + (22.610885561877026 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.25489083295460896 = (268.30952705895555 - 199.91988821724118) / 268.30952705895555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.DivisionOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 251.66644270960882 < 290.50531000585545.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 33.79274429302644 (T) = (0 -252.01437495052247) / Math.Sqrt((447.2662997011356 / (299)) + (25.606880027244383 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.16965890238650536 = (303.50704749511215 - 252.01437495052247) / 303.50704749511215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 235.17851458019894 < 260.2371025063681.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.637098617661554 (T) = (0 -238.77768222197977) / Math.Sqrt((241.38117294311496 / (299)) + (82.23893330187181 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14327981294026143 = (278.71139939104756 - 238.77768222197977) / 278.71139939104756 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.27762859981626 < 38.520425370216266.
IsChangePoint: Marked as a change because one of 9/20/2022 10:35:57 PM, 9/21/2022 6:35:49 PM, 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 5.807859761336816 (T) = (0 -38.25027303731115) / Math.Sqrt((2.8046687211288326 / (299)) + (3.358704054822022 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.05261369195527712 = (40.37452590617922 - 38.25027303731115) / 40.37452590617922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 271.97462864925984 < 279.2290290729863.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 19.38491591710684 (T) = (0 -262.7439406977711) / Math.Sqrt((103.75017888336684 / (299)) + (94.14801496104674 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1226980227486333 = (299.4908794357921 - 262.7439406977711) / 299.4908794357921 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.AbsBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 205.46328155346546 < 220.51111855719904.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.879070427911127 (T) = (0 -209.07611719069848) / Math.Sqrt((62.10511176816043 / (299)) + (17.845464077563655 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.12152093130584087 = (237.99783585223696 - 209.07611719069848) / 237.99783585223696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 242.8733475587968 < 267.8301946695548.
IsChangePoint: Marked as a change because one of 8/5/2022 12:58:40 PM, 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 21.706012158847333 (T) = (0 -247.93689914335468) / Math.Sqrt((381.97026212391563 / (299)) + (51.913288377601425 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13540659673987626 = (286.76704935344014 - 247.93689914335468) / 286.76704935344014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.DivideBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 249.56740900857756 < 297.2958175582705.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 25.42439090009683 (T) = (0 -253.3036384058843) / Math.Sqrt((742.5921903515421 / (299)) + (58.97956006227288 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.17698054083503265 = (307.77357155429263 - 253.3036384058843) / 307.77357155429263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 232.810977743561 < 261.82808121865816.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 21.277695940991787 (T) = (0 -238.5301901261424) / Math.Sqrt((38.317764030194965 / (299)) + (101.8720029450822 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.14150529027278622 = (277.84701224534655 - 238.5301901261424) / 277.84701224534655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.SquareRootBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 192.15788187578886 < 225.53872594680686.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 8.109985253528466 (T) = (0 -202.4269558089636) / Math.Sqrt((22.96325420348581 / (299)) + (482.2575537003863 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.14071210921149654 = (235.57524547822 - 202.4269558089636) / 235.57524547822 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.UnaryNegateOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 232.27821867016834 < 256.5385558399384.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 14.082388857593948 (T) = (0 -240.5100771035541) / Math.Sqrt((127.72629292371181 / (299)) + (129.79628150049905 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11320905081805463 = (271.21395107316096 - 240.5100771035541) / 271.21395107316096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.IO.Tests.Perf_Directory

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Exists - Duration of single invocation 3.79 μs 3.41 μs 0.90 0.02 False
MoveFiles - Duration of single invocation 12.52 μs 11.62 μs 0.93 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Directory*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_Directory.Exists

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.411523594685356 < 3.60281883653706.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 26.279260486448084 (T) = (0 -3444.3609179409727) / Math.Sqrt((7575.844322688371 / (299)) + (1840.244309692781 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06632204501855968 = (3689.02457165698 - 3444.3609179409727) / 3689.02457165698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Directory.MoveFiles

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.619454773402675 < 11.868785867857692.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 18.940452740741936 (T) = (0 -11668.597296629756) / Math.Sqrt((57006.27857579869 / (299)) + (25318.15736114368 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.05027789501833825 = (12286.32800629092 - 11668.597296629756) / 12286.32800629092 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractJsonSerializer_ - Duration of single invocation 43.62 μs 40.80 μs 0.94 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

MicroBenchmarks.Serializers.Json_ToStream<Location>.DataContractJsonSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.80389399795119 < 40.97350729643597.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 58.6199585685358 (T) = (0 -40619.90038630804) / Math.Sqrt((344387.0472583682 / (299)) + (61906.85859637831 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07641057064587288 = (43980.473460717105 - 40619.90038630804) / 43980.473460717105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBoolean - Duration of single invocation 2.35 μs 1.91 μs 0.81 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Get.GetBoolean

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.91076150006182 < 2.1483213770303795.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 33.76590281728582 (T) = (0 -1930.6557653446835) / Math.Sqrt((2734.9558258035067 / (299)) + (2198.4174749974827 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1407082744497785 = (2246.798971686183 - 1930.6557653446835) / 2246.798971686183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToDateTime_String - Duration of single invocation 2.94 μs 2.70 μs 0.92 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Perf_Convert.ToDateTime_String(value: "12/12/1999")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7000963848476327 < 2.7852729405503776.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.378254160875354 (T) = (0 -2710.464271648903) / Math.Sqrt((6986.76290152446 / (299)) + (2213.5528118392526 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08240436628930592 = (2953.8766010557074 - 2710.464271648903) / 2953.8766010557074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 9.02 μs 8.24 μs 0.91 0.01 False
DeserializeFromReader - Duration of single invocation 14.46 μs 13.52 μs 0.93 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.236285113076383 < 8.563219801179716.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 37.67996443437006 (T) = (0 -8287.791557378518) / Math.Sqrt((18835.942368324235 / (299)) + (9472.283932209168 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08465688119465839 = (9054.300389776583 - 8287.791557378518) / 9054.300389776583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;.DeserializeFromReader

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.516924029570559 < 13.776135304512321.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.013669815571017 (T) = (0 -13463.741570836155) / Math.Sqrt((120751.02316441268 / (119)) + (13230.529067641353 / (29))) is greater than 1.9763456545936318 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (119) + (29) - 2, .975) and 0.07633907475513475 = (14576.497936476935 - 13463.741570836155) / 14576.497936476935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in Benchstone.BenchF.SqMtx

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 4.72 secs 3.90 secs 0.83 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.SqMtx*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchF.SqMtx.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.9048759585333332 < 4.483654636235475.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 5.0717287976405085 (T) = (0 -3945582195.633131) / Math.Sqrt((1.535060579438203E+17 / (299)) + (80374099845809840 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0686175638385046 = (4236264334.008757 - 3945582195.633131) / 4236264334.008757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 2.62 μs 2.41 μs 0.92 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_DateTime.ToString(format: "s")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4113541002116237 < 2.508518480783025.
IsChangePoint: Marked as a change because one of 7/31/2022 5:42:25 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 33.75886882613844 (T) = (0 -2443.399447949094) / Math.Sqrt((2619.2270676759867 / (299)) + (1315.1581764006594 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09349877272486382 = (2695.4176943518764 - 2443.399447949094) / 2695.4176943518764 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Zip - Duration of single invocation 15.94 μs 14.48 μs 0.91 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.480307603092784 < 15.302578087754899.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.966686515585176 (T) = (0 -14541.450945598725) / Math.Sqrt((1132763.4298120046 / (299)) + (46998.03061193862 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.05735722748681739 = (15426.258355357375 - 14541.450945598725) / 15426.258355357375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFullPathForLegacyLength - Duration of single invocation 7.84 μs 6.48 μs 0.83 0.09 False
GetFullPathNoRedundantSegments - Duration of single invocation 4.25 μs 3.64 μs 0.86 0.06 False
GetFullPathWithRedundantSegments - Duration of single invocation 13.87 μs 12.56 μs 0.91 0.03 False
HasExtension - Duration of single invocation 85.17 ns 69.91 ns 0.82 0.15 False
GetFullPathForReallyLongPath - Duration of single invocation 37.93 μs 31.94 μs 0.84 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_Path.GetFullPathForLegacyLength

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.483871554420349 < 7.339796253323451.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 17.735047532302364 (T) = (0 -6550.13362961937) / Math.Sqrt((153621.75685345338 / (299)) + (34492.543956025605 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10162906549283747 = (7291.123719638937 - 6550.13362961937) / 7291.123719638937 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathNoRedundantSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6436487771007573 < 4.032458308348913.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 22.368981665530516 (T) = (0 -3617.845185026078) / Math.Sqrt((26850.927232389706 / (299)) + (6873.2417256533245 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10292815487378972 = (4032.9492054419325 - 3617.845185026078) / 4032.9492054419325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathWithRedundantSegments

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.563157148654918 < 13.181071188947833.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 42.285179883097946 (T) = (0 -12474.021822563986) / Math.Sqrt((49982.97628108122 / (299)) + (24992.968972651634 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09807502870135917 = (13830.442907687995 - 12474.021822563986) / 13830.442907687995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.HasExtension

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.90703056900578 < 79.20130641378641.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 51.26386138278077 (T) = (0 -70.48428163933424) / Math.Sqrt((9.737421033419773 / (299)) + (0.8508716270068284 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.15323318828422286 = (83.23930586806318 - 70.48428163933424) / 83.23930586806318 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathForReallyLongPath

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.944553340149966 < 36.29877834597504.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 28.07925503155663 (T) = (0 -32117.62566123155) / Math.Sqrt((2505232.9228334907 / (299)) + (304751.25722614035 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10726332691670597 = (35976.59492390419 - 32117.62566123155) / 35976.59492390419 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare - Duration of single invocation 34.70 ns 25.58 ns 0.74 0.23 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Enum.Compare

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.575824638076277 < 31.703768494021233.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.96641629532215 (T) = (0 -25.83883570941351) / Math.Sqrt((1.0178091403368694 / (299)) + (0.7282981498251878 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.19272135632017623 = (32.007332179174426 - 25.83883570941351) / 32.007332179174426 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 26.45 μs 24.00 μs 0.91 0.01 False
DeserializeFromString - Duration of single invocation 31.23 μs 28.73 μs 0.92 0.00 False
DeserializeFromReader - Duration of single invocation 38.49 μs 35.24 μs 0.92 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.998961786442404 < 24.67393078886656.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.006540728754956 (T) = (0 -24216.21336297935) / Math.Sqrt((97491.41557298464 / (299)) + (55878.17287774315 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07439484304892598 = (26162.57394540357 - 24216.21336297935) / 26162.57394540357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;.DeserializeFromString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.726236258629843 < 29.47592183417151.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 33.35248883372688 (T) = (0 -29157.341918142545) / Math.Sqrt((107924.39724437817 / (299)) + (110794.97722183669 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06887368016879751 = (31314.05621036282 - 29157.341918142545) / 31314.05621036282 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;.DeserializeFromReader

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.23984053531119 < 36.349536899629214.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.37782192566161 (T) = (0 -35716.077697871966) / Math.Sqrt((273974.98038546904 / (119)) + (107091.00695873317 / (29))) is greater than 1.9763456545936318 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (119) + (29) - 2, .975) and 0.07123166521986282 = (38455.31373152042 - 35716.077697871966) / 38455.31373152042 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFiveValues - Duration of single invocation 7.72 μs 3.23 μs 0.42 0.10 False
Fill - Duration of single invocation 2.00 μs 1.76 μs 0.88 0.18 False
IndexOfAnyThreeValues - Duration of single invocation 2.45 μs 1.36 μs 0.56 0.13 False
Reverse - Duration of single invocation 2.52 μs 1.97 μs 0.78 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2336507922474165 < 7.3384644693729415.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 48.31808560907212 (T) = (0 -3222.494685189797) / Math.Sqrt((757086.8161663241 / (90)) + (1283.4031993301635 / (29))) is greater than 1.9804475986831467 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (29) - 2, .975) and 0.5796240282831087 = (7665.744243250982 - 3222.494685189797) / 7665.744243250982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Fill(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.764605275430024 < 1.8990145087137311.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.551051405404625 (T) = (0 -1699.4099219788854) / Math.Sqrt((12391.649219022027 / (299)) + (1977.5030261153108 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1613342814844044 = (2026.3257272358435 - 1699.4099219788854) / 2026.3257272358435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3640026512355505 < 2.3839098454648626.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 25.46244429697399 (T) = (0 -1379.4599843174396) / Math.Sqrt((1000019.3465435071 / (299)) + (11470.432569041815 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.5302600102725428 = (2936.6458348964525 - 1379.4599843174396) / 2936.6458348964525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Reverse(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9681274975710727 < 2.1050045306984795.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 26.516888258103993 (T) = (0 -1968.3762448925033) / Math.Sqrt((4413.383666374489 / (299)) + (1690.007291704542 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.09969871961976343 = (2186.352821869999 - 1968.3762448925033) / 2186.352821869999 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 2.83 μs 2.62 μs 0.93 0.01 False
ToString - Duration of single invocation 9.02 μs 8.24 μs 0.91 0.00 False
ToString - Duration of single invocation 9.13 μs 8.25 μs 0.90 0.01 False
ToString - Duration of single invocation 2.99 μs 2.77 μs 0.93 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_DateTimeOffset.ToString(format: "s")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.623913518870099 < 2.6560716083145794.
IsChangePoint: Marked as a change because one of 7/29/2022 1:23:45 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 46.04676994357239 (T) = (0 -2594.2831846831004) / Math.Sqrt((3409.9542241714294 / (299)) + (603.7637917878773 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09153353114605907 = (2855.673020002456 - 2594.2831846831004) / 2855.673020002456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.ToString(format: null)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.2441581883676 < 8.547699016064778.
IsChangePoint: Marked as a change because one of 7/31/2022 5:42:25 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.056546661622235 (T) = (0 -8250.160382186084) / Math.Sqrt((30151.91872021497 / (299)) + (14275.416607685174 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09619646383033915 = (9128.26742983375 - 8250.160382186084) / 9128.26742983375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.ToString(value: 12/30/2017 3:45:22 AM -08:00)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.24977527297264 < 8.570203871492701.
IsChangePoint: Marked as a change because one of 7/31/2022 12:39:41 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 46.09270673918358 (T) = (0 -8304.431649078648) / Math.Sqrt((22921.579921226185 / (299)) + (8371.918596150113 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09476356052577888 = (9173.770837044543 - 8304.431649078648) / 9173.770837044543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTimeOffset.ToString(format: "G")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7740259955752213 < 2.777638125436431.
IsChangePoint: Marked as a change because one of 7/30/2022 3:39:54 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 26.734583127374112 (T) = (0 -2751.7530562362163) / Math.Sqrt((10729.077639148185 / (299)) + (2518.4537192841635 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09717137293226619 = (3047.9240176217504 - 2751.7530562362163) / 3047.9240176217504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Split - Duration of single invocation 575.28 ns 431.74 ns 0.75 0.05 False
PadLeft - Duration of single invocation 8.61 μs 7.35 μs 0.85 0.14 False
Replace_String - Duration of single invocation 1.01 μs 859.96 ns 0.86 0.06 False
CtorCharCount - Duration of single invocation 566.21 ns 507.62 ns 0.90 0.09 False
Replace_Char - Duration of single invocation 11.59 μs 10.05 μs 0.87 0.08 False
Split - Duration of single invocation 533.13 ns 446.72 ns 0.84 0.06 False
Replace_Char - Duration of single invocation 1.29 μs 1.14 μs 0.89 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 431.7409025657799 < 514.0926859257188.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 53.267866011943156 (T) = (0 -436.92497597139925) / Math.Sqrt((439.3943294313128 / (299)) + (42.473458279061 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1715603468560112 = (527.4071253267962 - 436.92497597139925) / 527.4071253267962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.PadLeft(n: 2142)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.353087055035127 < 8.197436706463487.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 22.661018855345414 (T) = (0 -7407.5542492761515) / Math.Sqrt((68891.61733925177 / (299)) + (90981.68621119494 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.1468394605130467 = (8682.485776628479 - 7407.5542492761515) / 8682.485776628479 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 859.955283126941 < 920.4803020465189.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 10.12355957414548 (T) = (0 -871.5067897963689) / Math.Sqrt((2732.643743733732 / (299)) + (334.81641564002894 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.050652190588827986 = (918.0057942482813 - 871.5067897963689) / 918.0057942482813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 507.6190244633786 < 537.8413682562427.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 32.76855872570915 (T) = (0 -510.20225366086936) / Math.Sqrt((815.1833717658602 / (299)) + (15.903026983254174 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1044026207656426 = (569.6781449908208 - 510.20225366086936) / 569.6781449908208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.054889563185043 < 10.994877138101204.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 14.446133427074553 (T) = (0 -10099.591284439122) / Math.Sqrt((468628.8871136181 / (299)) + (8957.74989190388 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05819105791129501 = (10723.60946376307 - 10099.591284439122) / 10723.60946376307 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 446.72005263276867 < 508.64374706170145.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 57.03550647581926 (T) = (0 -439.2677843306602) / Math.Sqrt((227.00478916953588 / (299)) + (52.496687816197515 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.16898334350567817 = (528.5908301570387 - 439.2677843306602) / 528.5908301570387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58l", oldChar: 'b', newChar: '+')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1442025351892036 < 1.2306623774456427.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 14.032106336966427 (T) = (0 -1156.5211192077643) / Math.Sqrt((7973.502142561234 / (299)) + (529.1074549993943 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07519711842820091 = (1250.5595973513143 - 1156.5211192077643) / 1250.5595973513143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsNaN - Duration of single invocation 7.57 ms 7.12 ms 0.94 0.02 False
TryParse - Duration of single invocation 812.20 ns 715.60 ns 0.88 0.05 False
TryParse - Duration of single invocation 1.58 μs 1.39 μs 0.88 0.01 False
ToStringWithCultureInfo - Duration of single invocation 3.65 μs 3.31 μs 0.91 0.08 False
TryParse - Duration of single invocation 1.59 μs 1.38 μs 0.87 0.02 False
Parse - Duration of single invocation 1.60 μs 1.40 μs 0.88 0.01 False
Parse - Duration of single invocation 807.82 ns 705.45 ns 0.87 0.03 False
ToStringWithFormat - Duration of single invocation 3.82 μs 3.33 μs 0.87 0.05 False
ToStringWithFormat - Duration of single invocation 2.63 μs 2.27 μs 0.86 0.28 False
ToStringWithFormat - Duration of single invocation 3.89 μs 3.41 μs 0.88 0.06 False
Parse - Duration of single invocation 1.61 μs 1.36 μs 0.85 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Double.IsNaN(value: 0)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.117175316666666 < 7.1813883984241444.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.240243584414486 (T) = (0 -7114679.777705027) / Math.Sqrt((49103214096.64962 / (299)) + (352326860.86709976 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06165445379813476 = (7582153.2979008285 - 7114679.777705027) / 7582153.2979008285 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "12345")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 715.5991242376969 < 762.7991918534753.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 22.74412984566202 (T) = (0 -718.0153665650448) / Math.Sqrt((628.976215298796 / (299)) + (156.9459420726292 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07901207281555368 = (779.614309125735 - 718.0153665650448) / 779.614309125735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3942365616053698 < 1.5358492854317658.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.344594526290685 (T) = (0 -1390.4879303883024) / Math.Sqrt((2559.1374094984126 / (299)) + (225.44903618877512 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.10579173525893673 = (1554.9933781823715 - 1390.4879303883024) / 1554.9933781823715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.311236084332777 < 3.43186740862233.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 18.23332429995885 (T) = (0 -3327.4658282943947) / Math.Sqrt((6044.323033012943 / (299)) + (5025.261683280982 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07082555831998258 = (3581.0991768973818 - 3327.4658282943947) / 3581.0991768973818 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3804762394691958 < 1.497373006072681.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.874109688493707 (T) = (0 -1368.2478099587802) / Math.Sqrt((1562.2997514108442 / (299)) + (892.8528100378377 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.11163741530140964 = (1540.1907211378207 - 1368.2478099587802) / 1540.1907211378207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4028295259778991 < 1.540275687246816.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.73009712458031 (T) = (0 -1397.1096343325755) / Math.Sqrt((2851.6536993884524 / (299)) + (308.9718897722368 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.10481759413779684 = (1560.6982724229667 - 1397.1096343325755) / 1560.6982724229667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "12345")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 705.4481022861963 < 769.2681221318221.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 22.33799806964772 (T) = (0 -720.1928810134498) / Math.Sqrt((902.2615373199784 / (299)) + (130.68440278477505 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.0791755852366159 = (782.1174910946635 - 720.1928810134498) / 782.1174910946635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.3304604355733223 < 3.649580194022636.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 19.99681452658803 (T) = (0 -3487.440228266462) / Math.Sqrt((4822.963037483796 / (299)) + (7994.682759392205 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08668945385070566 = (3818.4604820016907 - 3487.440228266462) / 3818.4604820016907 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2666652435249834 < 2.4018791377907367.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 23.756066310962876 (T) = (0 -2291.0180609897607) / Math.Sqrt((7207.796544641026 / (299)) + (2266.374342769934 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09606750179194522 = (2534.501265892584 - 2291.0180609897607) / 2534.501265892584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4115021821493623 < 3.709788273849647.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 23.989450907728028 (T) = (0 -3563.354413136196) / Math.Sqrt((5008.710149067556 / (299)) + (4494.27017749553 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.0835135897767005 = (3888.060284786977 - 3563.354413136196) / 3888.060284786977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3638653412873822 < 1.4908895645454903.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.97554905808094 (T) = (0 -1373.979029165072) / Math.Sqrt((1631.0391463451178 / (299)) + (479.12004567980136 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1080966224246411 = (1540.5021033783241 - 1373.979029165072) / 1540.5021033783241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 15.64 ms 14.06 ms 0.90 0.02 False
Parse - Duration of single invocation 141.75 μs 128.80 μs 0.91 0.04 False
Parse - Duration of single invocation 193.90 μs 170.32 μs 0.88 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: Json400KB)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.055564100840336 < 14.7530517009375.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 53.823015256502984 (T) = (0 -13965341.084256131) / Math.Sqrt((41417636539.95231 / (299)) + (20228414470.4112 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1015968437480795 = (15544626.025712805 - 13965341.084256131) / 15544626.025712805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 128.79692096994538 < 135.68855270263563.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.158868424504085 (T) = (0 -129019.30893483313) / Math.Sqrt((9054180.112246033 / (299)) + (5871776.797773326 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08980494038037115 = (141749.0762790455 - 129019.30893483313) / 141749.0762790455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 170.32325756987578 < 183.21001734266787.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.74973149962062 (T) = (0 -173424.76509285468) / Math.Sqrt((13969696.412211254 / (299)) + (9167644.621653263 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11172553410651423 = (195237.81415737586 - 173424.76509285468) / 195237.81415737586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 27.36 μs 25.22 μs 0.92 0.00 False
Parse - Duration of single invocation 19.27 ms 17.19 ms 0.89 0.01 False
Parse - Duration of single invocation 26.48 μs 24.21 μs 0.91 0.01 False
Parse - Duration of single invocation 31.50 μs 28.63 μs 0.91 0.00 False
Parse - Duration of single invocation 5.39 μs 4.95 μs 0.92 0.02 False
Parse - Duration of single invocation 5.03 μs 4.61 μs 0.92 0.04 False
Parse - Duration of single invocation 22.10 μs 20.58 μs 0.93 0.01 False
Parse - Duration of single invocation 15.40 ms 13.85 ms 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.219415354065696 < 25.98621720308832.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 49.59821072060932 (T) = (0 -25216.40864964238) / Math.Sqrt((86258.92441587221 / (299)) + (60743.54640461218 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08760639629612825 = (27637.64295066964 - 25216.40864964238) / 27637.64295066964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.189759329670327 < 18.334243530833334.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.91511410080462 (T) = (0 -17432777.108610973) / Math.Sqrt((59683293819.45796 / (299)) + (64846522578.57108 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10607581193322455 = (19501404.41586167 - 17432777.108610973) / 19501404.41586167 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.210028040428124 < 25.19437692003647.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 58.07758986582435 (T) = (0 -24477.544656469254) / Math.Sqrt((88653.09479539025 / (299)) + (44232.025741961814 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09078313474786874 = (26921.56909086974 - 24477.544656469254) / 26921.56909086974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.63071053240741 < 29.830830403595815.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 56.11955383762526 (T) = (0 -28752.370346456915) / Math.Sqrt((109846.10834056967 / (299)) + (72011.42132895433 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.09193099729034877 = (31663.199889722793 - 28752.370346456915) / 31663.199889722793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.946033316088296 < 5.208137548983253.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.86126283047209 (T) = (0 -4925.738742467008) / Math.Sqrt((6598.732592871116 / (299)) + (5725.5291947193 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.102740505831931 = (5489.759400132187 - 4925.738742467008) / 5489.759400132187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.613382062708703 < 4.754908235098347.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.927339388172275 (T) = (0 -4543.095054425787) / Math.Sqrt((7369.777781745721 / (299)) + (8294.091708704334 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09775495835780809 = (5035.322827772841 - 4543.095054425787) / 5035.322827772841 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.576028893849205 < 21.088205723459577.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 48.33833514174716 (T) = (0 -20642.029384772806) / Math.Sqrt((61948.0335744176 / (299)) + (44181.70429909518 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08757481892247476 = (22623.257021902526 - 20642.029384772806) / 22623.257021902526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.854433659259257 < 14.58230589125.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 56.315848465549934 (T) = (0 -13920333.417642955) / Math.Sqrt((37915810977.632324 / (299)) + (20889890092.63163 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.10399830413848846 = (15536056.998484207 - 13920333.417642955) / 15536056.998484207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same - Duration of single invocation 13.97 μs 12.07 μs 0.86 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.06791064993565 < 13.25899423198901.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 172.34355824369908 (T) = (0 -12055.178682709104) / Math.Sqrt((23176.816572289426 / (299)) + (1690.0620979270677 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1419240683441272 = (14049.081483320027 - 12055.178682709104) / 14049.081483320027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 343.80 μs 314.10 μs 0.91 0.00 False
ReadLineAsync - Duration of single invocation 271.10 μs 223.00 μs 0.82 0.01 False
ReadLineAsync - Duration of single invocation 236.64 μs 206.54 μs 0.87 0.01 False
ReadLine - Duration of single invocation 418.64 μs 379.85 μs 0.91 0.03 False
ReadLine - Duration of single invocation 236.59 μs 197.63 μs 0.84 0.04 False
ReadLineAsync - Duration of single invocation 248.81 μs 217.66 μs 0.87 0.00 False
ReadLine - Duration of single invocation 234.51 μs 198.27 μs 0.85 0.04 False
ReadLine - Duration of single invocation 233.46 μs 197.94 μs 0.85 0.02 False
ReadLine - Duration of single invocation 260.65 μs 228.55 μs 0.88 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128])

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 314.1048535896502 < 325.0621139288949.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 69.98226904485215 (T) = (0 -313964.2628824758) / Math.Sqrt((9553644.364136662 / (299)) + (4641564.166224767 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09017635356760102 = (345082.5488143693 - 313964.2628824758) / 345082.5488143693 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 222.9966756944445 < 241.6660797091014.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 63.624160495286816 (T) = (0 -223940.7070128872) / Math.Sqrt((10383738.352185419 / (299)) + (7117435.846399951 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1290473065060643 = (257121.55055692064 - 223940.7070128872) / 257121.55055692064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 206.5353523769305 < 225.90056475773358.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 82.72332207505416 (T) = (0 -206388.81508661932) / Math.Sqrt((10285898.052306468 / (299)) + (3810119.749546348 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.14030081751812226 = (240070.96818539774 - 206388.81508661932) / 240070.96818539774 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   9,   32])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 379.8537744472789 < 391.80771462053576.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 62.594739453404586 (T) = (0 -379679.50342570443) / Math.Sqrt((28818878.54411378 / (299)) + (7164398.277364797 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.08912665642324415 = (416830.1840240539 - 379679.50342570443) / 416830.1840240539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.6317060825893 < 225.1107978826898.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 56.796686981069286 (T) = (0 -198969.4813114777) / Math.Sqrt((30556308.80105121 / (299)) + (8608333.312813492 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.15627513047356487 = (235822.70536028533 - 198969.4813114777) / 235822.70536028533 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 217.656690258216 < 236.29427603753308.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 100.35438639543193 (T) = (0 -218820.45756455726) / Math.Sqrt((10758023.564979987 / (299)) + (2215440.3410715098 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.13325238527192704 = (252461.563027443 - 218820.45756455726) / 252461.563027443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   0, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 198.27001491862563 < 222.56579233519903.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 68.00797745019405 (T) = (0 -198386.464968406) / Math.Sqrt((14739351.729712388 / (299)) + (6562388.826172649 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1525134763450287 = (234088.04674889805 - 198386.464968406) / 234088.04674889805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.94425182570592 < 223.57463510183456.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 85.61365722195418 (T) = (0 -198653.13721896565) / Math.Sqrt((10813540.119563362 / (299)) + (4090924.251102973 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1535710183793111 = (234695.57580436004 - 198653.13721896565) / 234695.57580436004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 228.54671068840577 < 249.30684995406938.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 62.220238790329695 (T) = (0 -228184.28649350026) / Math.Sqrt((25147991.77830701 / (299)) + (7307383.110768231 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.135015463944392 = (263801.57908260083 - 228184.28649350026) / 263801.57908260083 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 26.01 μs 23.07 μs 0.89 0.01 False
Ctor_ByteArray - Duration of single invocation 80.12 ns 73.29 ns 0.91 0.17 False
Parse - Duration of single invocation 2.44 μs 2.10 μs 0.86 0.19 False
ToStringX - Duration of single invocation 4.24 μs 3.92 μs 0.92 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.070556139380532 < 24.675007162164164.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 70.07374494633844 (T) = (0 -23116.910688598382) / Math.Sqrt((161761.7738990846 / (299)) + (25008.199989683868 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10197604330670916 = (25741.97549664444 - 23116.910688598382) / 25741.97549664444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 123)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.29074084289073 < 78.35851292241763.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 12.479811471439211 (T) = (0 -74.93022441836592) / Math.Sqrt((10.814398768040643 / (299)) + (10.903759848148857 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09800537804098362 = (83.07169754031028 - 74.93022441836592) / 83.07169754031028 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.101128586868414 < 2.247761739746303.
IsChangePoint: Marked as a change because one of 8/1/2022 4:16:36 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.107270471225302 (T) = (0 -2102.409041831214) / Math.Sqrt((5352.185223933817 / (299)) + (1021.2171525615013 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09836986290385927 = (2331.786566720578 - 2102.409041831214) / 2331.786566720578 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.9182520445979905 < 4.0275802008527.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 34.65816167683746 (T) = (0 -3971.5598770159286) / Math.Sqrt((8666.81567177309 / (299)) + (1511.8256261349752 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0721410479217458 = (4280.348719080929 - 3971.5598770159286) / 4280.348719080929 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 13.37 μs 10.71 μs 0.80 0.15 False
IndexOf_Word_NotFound - Duration of single invocation 8.05 μs 7.45 μs 0.93 0.10 False
LastIndexOf_Word_NotFound - Duration of single invocation 8.31 μs 7.43 μs 0.89 0.10 False
LastIndexOf_Word_NotFound - Duration of single invocation 25.43 μs 23.12 μs 0.91 0.06 False
LastIndexOf_Word_NotFound - Duration of single invocation 13.09 μs 11.03 μs 0.84 0.18 False
IndexOf_Word_NotFound - Duration of single invocation 13.21 μs 10.71 μs 0.81 0.13 False
IndexOf_Word_NotFound - Duration of single invocation 14.03 μs 10.67 μs 0.76 0.15 False
LastIndexOf_Word_NotFound - Duration of single invocation 25.55 μs 23.04 μs 0.90 0.06 False
LastIndexOf_Word_NotFound - Duration of single invocation 8.25 μs 7.50 μs 0.91 0.08 False
LastIndexOf_Word_NotFound - Duration of single invocation 8.21 μs 7.16 μs 0.87 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.707601827356457 < 12.45591357094577.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 46.40118410707329 (T) = (0 -10787.651515329702) / Math.Sqrt((162817.64241831005 / (299)) + (54665.57177589344 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.17304467010207597 = (13045.023262213312 - 10787.651515329702) / 13045.023262213312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.454079108764682 < 7.638813378206064.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 31.562378204517987 (T) = (0 -7467.451853563551) / Math.Sqrt((101368.59474470747 / (299)) + (2371.439817268733 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08031171411748496 = (8119.546555274354 - 7467.451853563551) / 8119.546555274354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.433610976723864 < 7.622499174320104.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.311920218440655 (T) = (0 -7458.535334883426) / Math.Sqrt((81202.31769349033 / (299)) + (1285.3941327792686 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07946826708112004 = (8102.420664232218 - 7458.535334883426) / 8102.420664232218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, True))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.124544563609465 < 24.24355155604276.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 60.50064378510234 (T) = (0 -23091.725743670708) / Math.Sqrt((227604.41176205498 / (299)) + (20487.180159205924 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09192645963403784 = (25429.356453183875 - 23091.725743670708) / 25429.356453183875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.033231381934218 < 12.430661091965524.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 54.11250987436446 (T) = (0 -10794.205392759877) / Math.Sqrt((218332.70250387755 / (299)) + (29365.287725538674 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.17306290101276753 = (13053.236341651344 - 10794.205392759877) / 13053.236341651344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.71084434339968 < 12.50515146088472.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 48.573888187920126 (T) = (0 -10751.186424391264) / Math.Sqrt((175449.60659372335 / (299)) + (50771.20838336494 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.1756609676570037 = (13042.190170024414 - 10751.186424391264) / 13042.190170024414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.671688731190153 < 12.487457192682339.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 37.51226957142854 (T) = (0 -10759.632173694888) / Math.Sqrt((164609.00136081156 / (299)) + (89940.16009033755 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1740222204488184 = (13026.53950272299 - 10759.632173694888) / 13026.53950272299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, True))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.042960865290066 < 24.203760203880872.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 37.10072216609121 (T) = (0 -23175.39459785175) / Math.Sqrt((304985.23672190565 / (299)) + (81713.74959403115 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08921314160176731 = (25445.46441810707 - 23175.39459785175) / 25445.46441810707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.496814017985243 < 7.630265954434905.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.237637491030178 (T) = (0 -7502.4552358309265) / Math.Sqrt((35213.39138734211 / (299)) + (80118.77848631212 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07440871824423478 = (8105.5811390092495 - 7502.4552358309265) / 8105.5811390092495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.164166626730564 < 7.778784979153449.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.28795541197238 (T) = (0 -7144.63701298403) / Math.Sqrt((101395.28455678784 / (299)) + (17517.387158206813 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14269859636280144 = (8333.86832527288 - 7144.63701298403) / 8333.86832527288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 21.48 μs 20.15 μs 0.94 0.01 False
DeserializeFromUtf8Bytes - Duration of single invocation 17.30 μs 15.66 μs 0.91 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.150117584453245 < 20.46097245697313.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 40.979834264101406 (T) = (0 -20013.546918608674) / Math.Sqrt((95905.53537895548 / (299)) + (43457.99450936414 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07929974280455561 = (21737.310011807935 - 20013.546918608674) / 21737.310011807935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.659425239174215 < 16.33037580180326.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 37.1738747469798 (T) = (0 -15965.300438656745) / Math.Sqrt((101960.18587314861 / (299)) + (48890.6616608976 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.09253939171380193 = (17593.381236468562 - 15965.300438656745) / 17593.381236468562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 6.51 ms 3.80 ms 0.58 0.06 False
Count - Duration of single invocation 7.36 ms 4.57 ms 0.62 0.05 False
Count - Duration of single invocation 6.47 ms 3.83 ms 0.59 0.06 False
Count - Duration of single invocation 37.77 ms 35.02 ms 0.93 0.03 False
Count - Duration of single invocation 42.72 ms 31.69 ms 0.74 0.06 False
Count - Duration of single invocation 56.34 ms 47.12 ms 0.84 0.02 False
Count - Duration of single invocation 26.07 ms 24.07 ms 0.92 0.05 False
Count - Duration of single invocation 6.34 ms 3.84 ms 0.61 0.07 False
Count - Duration of single invocation 6.36 ms 3.83 ms 0.60 0.07 False
Count - Duration of single invocation 9.57 ms 6.87 ms 0.72 0.03 False
Count - Duration of single invocation 7.23 ms 4.60 ms 0.64 0.07 False
Count - Duration of single invocation 6.45 ms 3.82 ms 0.59 0.06 False
Count - Duration of single invocation 6.45 ms 3.74 ms 0.58 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7960084830769234 < 6.116088908589743.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.55304739122399 (T) = (0 -3782158.247980263) / Math.Sqrt((5317675826117.906 / (299)) + (1092522047.2148266 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.49304363039935556 = (7460520.223781118 - 3782158.247980263) / 7460520.223781118 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.568827623456789 < 6.963965016684174.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.287056012243628 (T) = (0 -4575806.621836544) / Math.Sqrt((6062649016839.725 / (299)) + (1090415157.0276873 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.4594411699423054 = (8464955.833480995 - 4575806.621836544) / 8464955.833480995 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.830275909598214 < 6.124709621538461.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.16436931621394 (T) = (0 -3790349.3391896435) / Math.Sqrt((5374081907897.692 / (299)) + (1527500523.525394 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.4903719900872046 = (7437482.370402338 - 3790349.3391896435) / 7437482.370402338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.01550112244899 < 35.65763076285714.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.84613998506451 (T) = (0 -35058256.49224184) / Math.Sqrt((827934166592.8718 / (299)) + (1044582924135.5428 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.062396560899954726 = (37391348.01584384 - 35058256.49224184) / 37391348.01584384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.687923785714286 < 40.58178472176587.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.366191446038467 (T) = (0 -32231121.221513607) / Math.Sqrt((23236542508519.5 / (299)) + (177950597837.57584 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09265417832127897 = (35522422.04838875 - 32231121.221513607) / 35522422.04838875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.11806623333334 < 54.128726189702384.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 9.518179834142025 (T) = (0 -47367992.76758851) / Math.Sqrt((22659673442052.836 / (299)) + (186446420684.15094 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.054409918402379985 = (50093580.389039196 - 47367992.76758851) / 50093580.389039196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.068618592857145 < 24.79001030720899.
IsChangePoint: Marked as a change because one of 7/15/2022 2:19:08 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 7.2715043237458135 (T) = (0 -24503588.39952766) / Math.Sqrt((271645746716.62845 / (299)) + (2175586452975.3738 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07681022233206894 = (26542309.059602194 - 24503588.39952766) / 26542309.059602194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.8381936223958335 < 6.083540247069596.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.417538929699234 (T) = (0 -3810969.150783844) / Math.Sqrt((5264380639590.432 / (299)) + (1762567335.0882237 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.4888081309012706 = (7455066.054753328 - 3810969.150783844) / 7455066.054753328 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.829868813701923 < 6.116459004481982.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.347373389223062 (T) = (0 -3822511.263460394) / Math.Sqrt((5242055295776.189 / (299)) + (695129481.8014208 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.48663594956737266 = (7446004.955428898 - 3822511.263460394) / 7446004.955428898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.87253284814815 < 9.056322696004763.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.816875979495855 (T) = (0 -6895539.236373073) / Math.Sqrt((5322750978088.736 / (299)) + (3246243424.370823 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.350594181990029 = (10618228.30214804 - 6895539.236373073) / 10618228.30214804 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.604898155270655 < 6.8985752065756305.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.65797938966006 (T) = (0 -4627958.968317023) / Math.Sqrt((5340767176985.632 / (299)) + (2567092670.9761047 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.44466087781720676 = (8333572.736828906 - 4627958.968317023) / 8333572.736828906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.823286454545454 < 6.109833482411643.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.331931842264666 (T) = (0 -3887684.9816309013) / Math.Sqrt((5332119192007.03 / (299)) + (104256170046.06328 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.47742742179104203 = (7439512.029037919 - 3887684.9816309013) / 7439512.029037919 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7411898035502955 < 6.131980433333332.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 27.62808894658457 (T) = (0 -3794979.7784281583) / Math.Sqrt((5303648518049.015 / (299)) + (1157493390.5749223 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.492584684866234 = (7479040.669924827 - 3794979.7784281583) / 7479040.669924827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 666.31 ms 620.96 ms 0.93 0.09 False
WriteStringsUtf16 - Duration of single invocation 677.72 ms 627.38 ms 0.93 0.10 False
WriteStringsUtf8 - Duration of single invocation 536.44 ms 480.64 ms 0.90 0.15 False
WriteStringsUtf8 - Duration of single invocation 534.65 ms 488.50 ms 0.91 0.15 False
WriteStringsUtf16 - Duration of single invocation 661.73 ms 621.06 ms 0.94 0.09 False
WriteStringsUtf8 - Duration of single invocation 539.62 ms 498.13 ms 0.92 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: AllEscaped)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 620.955128 < 636.0146327376923.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 18.487973236989443 (T) = (0 -620576781.1608485) / Math.Sqrt((320447921124895.44 / (299)) + (285634957947352.3 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08839131950399255 = (680749091.6203122 - 620576781.1608485) / 680749091.6203122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 627.3751318 < 647.9402420407142.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.857802965469364 (T) = (0 -630782355.4861121) / Math.Sqrt((364614629138714.3 / (299)) + (360167373036283.44 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08983263602894798 = (693039962.1603816 - 630782355.4861121) / 693039962.1603816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 480.6363935333334 < 500.4501099597527.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 23.49154622084686 (T) = (0 -484103788.5384288) / Math.Sqrt((296446210597693.2 / (299)) + (116887139005536.11 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.0993373268825119 = (537497337.2248094 - 484103788.5384288) / 537497337.2248094 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 488.50443853333337 < 508.9745541358974.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 26.559416772467646 (T) = (0 -493690443.63104707) / Math.Sqrt((305841627312428.6 / (299)) + (90594446869418.88 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09873565714973542 = (547775408.5663061 - 493690443.63104707) / 547775408.5663061 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 621.0554457999999 < 631.4930640279487.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.20393998869295 (T) = (0 -624009845.411671) / Math.Sqrt((374119036241476.2 / (299)) + (145466036571490.44 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08850899496358088 = (684603404.7113152 - 624009845.411671) / 684603404.7113152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: AllEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 498.1270576666667 < 510.08030142794865.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.03527652161354 (T) = (0 -491438261.3256541) / Math.Sqrt((409941872018531.2 / (299)) + (108876531474377.6 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10085172153791382 = (546559753.3770691 - 491438261.3256541) / 546559753.3770691 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Depth

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSpanEmptyLoop - Duration of single invocation 308.70 μs 280.03 μs 0.91 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 280.03068590561224 < 289.02786036341615.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.354455213508018 (T) = (0 -284636.7582941037) / Math.Sqrt((57003814.32563721 / (299)) + (24692841.833635654 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08033028947088793 = (309498.89404354105 - 284636.7582941037) / 309498.89404354105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SameIndex2 - Duration of single invocation 15.40 μs 14.35 μs 0.93 0.04 False
ReadOnlyIndexer2 - Duration of single invocation 9.99 μs 9.29 μs 0.93 0.04 False
Indexer5 - Duration of single invocation 10.94 μs 9.96 μs 0.91 0.10 False
Fixed2 - Duration of single invocation 7.01 μs 6.34 μs 0.90 0.05 False
CoveredIndex3 - Duration of single invocation 21.41 μs 19.21 μs 0.90 0.13 False
KnownSizeCtor - Duration of single invocation 10.00 μs 9.31 μs 0.93 0.04 False
Fixed1 - Duration of single invocation 7.05 μs 6.34 μs 0.90 0.03 False
KnownSizeCtor2 - Duration of single invocation 10.05 μs 9.34 μs 0.93 0.03 False
Indexer2 - Duration of single invocation 10.02 μs 9.25 μs 0.92 0.05 False
SameIndex1 - Duration of single invocation 14.37 μs 12.46 μs 0.87 0.07 False
WriteViaIndexer2 - Duration of single invocation 13.23 μs 10.76 μs 0.81 0.06 False
ReadOnlyIndexer1 - Duration of single invocation 8.24 μs 7.58 μs 0.92 0.06 False
Indexer3 - Duration of single invocation 10.05 μs 9.30 μs 0.92 0.06 False
Indexer1 - Duration of single invocation 8.20 μs 7.62 μs 0.93 0.03 False
Ref - Duration of single invocation 7.05 μs 6.34 μs 0.90 0.04 False
CoveredIndex2 - Duration of single invocation 15.43 μs 13.53 μs 0.88 0.08 False
KnownSizeArray - Duration of single invocation 10.00 μs 9.30 μs 0.93 0.04 False
Indexer4 - Duration of single invocation 81.89 μs 75.62 μs 0.92 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Span.IndexerBench.SameIndex2(length: 1024)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.350431524410222 < 14.624551214470623.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 17.393520016847244 (T) = (0 -14363.548435983726) / Math.Sqrt((711042.1074069669 / (299)) + (57982.59155558609 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07473386809426132 = (15523.694146677153 - 14363.548435983726) / 15523.694146677153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.ReadOnlyIndexer2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.289809817181926 < 9.511909522230939.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 18.873614610410236 (T) = (0 -9294.278885736037) / Math.Sqrt((500874.31184787146 / (299)) + (2157.854805893998 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07834370012376533 = (10084.321983134198 - 9294.278885736037) / 10084.321983134198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer5(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.962086241236454 < 10.394867696020336.
IsChangePoint: Marked as a change because one of 6/28/2022 9:06:37 PM, 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.22152957664833 (T) = (0 -10024.656633910141) / Math.Sqrt((87808.03038476089 / (299)) + (8995.509610572068 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07947714381836363 = (10890.176780066924 - 10024.656633910141) / 10890.176780066924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Fixed2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.342661196963536 < 6.682651616120191.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.399055929992635 (T) = (0 -6344.357472202976) / Math.Sqrt((58868.307579694694 / (299)) + (8341.042281492319 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1084121189764002 = (7115.795994130437 - 6344.357472202976) / 7115.795994130437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.CoveredIndex3(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.21413188335679 < 20.33266380233579.
IsChangePoint: Marked as a change because one of 7/13/2022 4:49:41 PM, 7/14/2022 1:35:19 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 15.734289886033563 (T) = (0 -19322.034902104806) / Math.Sqrt((2574271.60925094 / (299)) + (393618.1805522778 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10921179849017085 = (21690.94165072594 - 19322.034902104806) / 21690.94165072594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.KnownSizeCtor(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.311657975124378 < 9.514185876915256.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 23.00846127093515 (T) = (0 -9302.639680719527) / Math.Sqrt((336791.5652614804 / (299)) + (307.3288072718315 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07696782325396827 = (10078.348203975025 - 9302.639680719527) / 10078.348203975025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Fixed1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.338730972673161 < 6.696180366874104.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 21.533545275830424 (T) = (0 -6368.141624712421) / Math.Sqrt((27468.61266058356 / (299)) + (30397.295138816495 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10396897003530997 = (7107.054791354012 - 6368.141624712421) / 7107.054791354012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.KnownSizeCtor2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.339991419764447 < 9.549039981787422.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 57.60940582918565 (T) = (0 -9334.474850288596) / Math.Sqrt((36081.49040017577 / (299)) + (1301.496493347173 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07325875124371059 = (10072.36363204476 - 9334.474850288596) / 10072.36363204476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.254005654408857 < 9.509191995495021.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 19.705724694529575 (T) = (0 -9290.477429875029) / Math.Sqrt((487773.68036710937 / (299)) + (413.6655020843651 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07925004363377584 = (10090.119869829006 - 9290.477429875029) / 10090.119869829006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.SameIndex1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.461046013471176 < 13.634243454179524.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 14.456661470256337 (T) = (0 -12593.889084563634) / Math.Sqrt((350870.6753519513 / (299)) + (353317.6116093883 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.11712731653237653 = (14264.67181553191 - 12593.889084563634) / 14264.67181553191 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.WriteViaIndexer2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.759295560720844 < 12.566622859923335.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 114.15177254806935 (T) = (0 -10762.824783719094) / Math.Sqrt((90574.80389577987 / (299)) + (6047.450265931601 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.19345868194985485 = (13344.418373678329 - 10762.824783719094) / 13344.418373678329 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.ReadOnlyIndexer1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.577157698220065 < 7.837431562885393.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 7.67439709594243 (T) = (0 -7715.823209848981) / Math.Sqrt((66826.31931020264 / (299)) + (162563.25548905117 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07057912211493075 = (8301.753697858192 - 7715.823209848981) / 8301.753697858192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer3(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.295049565291508 < 9.530098223029903.
IsChangePoint: Marked as a change because one of 7/9/2022 11:03:07 AM, 7/9/2022 7:52:53 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.50172885817578 (T) = (0 -9343.700316711253) / Math.Sqrt((607813.4038915151 / (299)) + (80238.04828344614 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0785791029258595 = (10140.534414165157 - 9343.700316711253) / 10140.534414165157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer1(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.61667120786517 < 7.8394745111875315.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 45.53314965536161 (T) = (0 -7616.952870099447) / Math.Sqrt((62720.139552952474 / (299)) + (316.9863136418582 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08156312695939727 = (8293.387486591811 - 7616.952870099447) / 8293.387486591811 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Ref(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.3391342517283435 < 6.69663583220098.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 59.123335412348354 (T) = (0 -6332.816624379171) / Math.Sqrt((47042.40804017866 / (299)) + (478.64783670432047 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10960189374377653 = (7112.342872118398 - 6332.816624379171) / 7112.342872118398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.CoveredIndex2(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.532561804861807 < 14.653123512243317.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.135734803336213 (T) = (0 -13630.646784718027) / Math.Sqrt((161575.96229674784 / (299)) + (123346.29606152701 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.10775663532021332 = (15276.82617131019 - 13630.646784718027) / 15276.82617131019 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.KnownSizeArray(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.295697686011906 < 9.509574935519336.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 19.173029787658777 (T) = (0 -9291.567775584965) / Math.Sqrt((525340.7641738213 / (299)) + (284.68916936490206 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07980603876719748 = (10097.401381700944 - 9291.567775584965) / 10097.401381700944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.IndexerBench.Indexer4(length: 1024)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 75.61917294254658 < 77.80199867642774.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 65.82319264724953 (T) = (0 -75679.59556646613) / Math.Sqrt((2908821.8801375837 / (299)) + (13609.224541248923 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.08063018459047416 = (82316.81560347429 - 75679.59556646613) / 82316.81560347429 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 1.03 μs 920.17 ns 0.89 0.05 False
TryParse - Duration of single invocation 997.42 ns 892.22 ns 0.89 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Decimal.Parse(value: "123456.789")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 920.1689428604038 < 0.9861817424563796.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 37.7024132582761 (T) = (0 -921.8431079836929) / Math.Sqrt((412.8088684492298 / (299)) + (193.45585255150806 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10535043053649971 = (1030.3957431472302 - 921.8431079836929) / 1030.3957431472302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.TryParse(value: "123456.789")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 892.2178691696096 < 0.9749044237528133.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 24.25109401282917 (T) = (0 -914.6694052899342) / Math.Sqrt((393.81532343335357 / (299)) + (466.31521306106845 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09957518225023397 = (1015.8198522068355 - 914.6694052899342) / 1015.8198522068355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CaseInsensitiveNotMatching - Duration of single invocation 31.18 μs 29.10 μs 0.93 0.00 False
Baseline - Duration of single invocation 30.92 μs 29.08 μs 0.94 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveNotMatching

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.09811789199798 < 29.345488324988622.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 44.381791311005834 (T) = (0 -29075.687483831454) / Math.Sqrt((102030.08751504787 / (299)) + (61062.1790347936 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.06836286168737978 = (31209.240473703416 - 29075.687483831454) / 31209.240473703416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive&lt;Location&gt;.Baseline

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.077382143967665 < 29.341550703606703.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.89565287650799 (T) = (0 -29027.302706441067) / Math.Sqrt((132103.27027608675 / (299)) + (82033.25089182872 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07117670244575634 = (31251.695325553417 - 29027.302706441067) / 31251.695325553417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ProducerConsumer - Duration of single invocation 4.55 μs 4.28 μs 0.94 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.2816455745 < 4.3521783999500006.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 30.287538196646945 (T) = (0 -4297.751765317604) / Math.Sqrt((5784.45743659533 / (299)) + (2465.7123661404253 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06716117714969044 = (4607.1750660909765 - 4297.751765317604) / 4607.1750660909765 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a7d80c177a7ea412fb00867ccc4f4b2c44c9e030
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Segment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 1.06 ms 964.17 μs 0.91 0.01 False
ReadMultiSegmentSequence - Duration of single invocation 1.21 ms 1.13 ms 0.93 0.01 False
ReadMultiSegmentSequence - Duration of single invocation 12.42 ms 11.55 ms 0.93 0.01 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 112.63 μs 103.91 μs 0.92 0.02 False
ReadSingleSegmentSequenceByN - Duration of single invocation 10.57 ms 9.45 ms 0.89 0.02 False
ReadMultiSegmentSequence - Duration of single invocation 1.25 ms 1.15 ms 0.92 0.01 False
ReadSingleSegmentSequenceByN - Duration of single invocation 107.93 μs 98.46 μs 0.91 0.01 False
ReadSingleSegmentSequence - Duration of single invocation 106.51 μs 96.99 μs 0.91 0.01 False
ReadMultiSegmentSequence - Duration of single invocation 12.15 ms 11.28 ms 0.93 0.01 False
ReadSingleSegmentSequenceByN - Duration of single invocation 1.02 ms 944.89 μs 0.92 0.01 False
ReadSingleSegmentSequenceByN - Duration of single invocation 106.03 μs 96.42 μs 0.91 0.01 False
ReadSingleSegmentSequenceByN - Duration of single invocation 1.06 ms 947.57 μs 0.90 0.01 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 10.53 ms 9.74 ms 0.93 0.01 False
ReadMultiSegmentSequence - Duration of single invocation 109.34 μs 99.30 μs 0.91 0.02 False
ReadMultiSegmentSequence - Duration of single invocation 128.42 μs 119.89 μs 0.93 0.01 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 1.08 ms 996.07 μs 0.92 0.01 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 115.36 μs 104.70 μs 0.91 0.01 False
ReadSingleSegmentSequenceByN - Duration of single invocation 10.58 ms 9.50 ms 0.90 0.01 False
ReadSingleSegmentSequence - Duration of single invocation 10.23 ms 9.37 ms 0.92 0.01 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 10.87 ms 9.80 ms 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Segment*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json40KB)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 964.17125122549 < 1.0043829746354167.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 39.20740053798724 (T) = (0 -968432.8977262891) / Math.Sqrt((167988576.5558008 / (299)) + (185957754.38684413 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.09395420179363331 = (1068856.4525583868 - 968432.8977262891) / 1068856.4525583868 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.127520946747449 < 1.1573300698494735.
IsChangePoint: Marked as a change because one of 8/1/2022 12:35:11 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 40.40870936907856 (T) = (0 -1141606.4378958477) / Math.Sqrt((376373996.8156915 / (299)) + (182677187.85579833 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08868128832287674 = (1252697.2433112012 - 1141606.4378958477) / 1252697.2433112012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.54991018095238 < 11.78408224330357.
IsChangePoint: Marked as a change because one of 8/1/2022 8:26:50 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 37.15351881003218 (T) = (0 -11612976.998599464) / Math.Sqrt((40445349993.945404 / (299)) + (21573717606.39287 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.0878291757495926 = (12731142.774866357 - 11612976.998599464) / 12731142.774866357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 103.90522355248918 < 105.12931557988222.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.23106816724512 (T) = (0 -102481.18622140602) / Math.Sqrt((2469393.5413531675 / (299)) + (3063119.7863591076 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.08911879007795986 = (112507.73987332235 - 102481.18622140602) / 112507.73987332235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.454818392307693 < 9.882647357083334.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.54593609285009 (T) = (0 -9551845.808585955) / Math.Sqrt((19944070788.31024 / (299)) + (14255994930.935024 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09319342825537649 = (10533498.660258897 - 9551845.808585955) / 10533498.660258897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1502088954081633 < 1.1773098583413462.
IsChangePoint: Marked as a change because one of 7/30/2022 2:52:48 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 53.04392876097127 (T) = (0 -1155692.4207358167) / Math.Sqrt((354278286.5422186 / (299)) + (102923617.50122629 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08975691579187542 = (1269652.5145711196 - 1155692.4207358167) / 1269652.5145711196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.46463050314465 < 103.03354116056035.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 42.83017231632696 (T) = (0 -99222.67033314743) / Math.Sqrt((1988831.2590835027 / (299)) + (1445275.1350312983 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09304739693454533 = (109402.2664445526 - 99222.67033314743) / 109402.2664445526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.99189400720162 < 101.2294200681364.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 47.99352793925115 (T) = (0 -97918.80458023396) / Math.Sqrt((2415262.1728976374 / (299)) + (1042520.2435186566 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09576528555039271 = (108289.14552327878 - 97918.80458023396) / 108289.14552327878 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.278798068181818 < 11.755358169517857.
IsChangePoint: Marked as a change because one of 7/30/2022 2:52:48 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.83627836460373 (T) = (0 -11476370.105580414) / Math.Sqrt((38991823023.60018 / (299)) + (21028748537.480595 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0858260929549835 = (12553814.998588977 - 11476370.105580414) / 12553814.998588977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 944.8937143382353 < 0.9878317790798611.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 50.434814035523914 (T) = (0 -947971.5225094726) / Math.Sqrt((199322795.9931099 / (299)) + (97551409.81811745 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09779087460872103 = (1050722.6050261317 - 947971.5225094726) / 1050722.6050261317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.4243107638889 < 100.82148553023448.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.534138406143164 (T) = (0 -97873.81871716752) / Math.Sqrt((1868419.631864237 / (299)) + (1809514.7479139373 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09350835207197901 = (107969.90677286316 - 97873.81871716752) / 107969.90677286316 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 947.5727757352942 < 1.0005536869035216.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 26.465090405499772 (T) = (0 -957972.4514501756) / Math.Sqrt((192028425.76832968 / (299)) + (417819835.6760976 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09679889872787208 = (1060641.3677982723 - 957972.4514501756) / 1060641.3677982723 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.742392621538462 < 10.1092973573706.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 44.88151900405375 (T) = (0 -9728200.501009095) / Math.Sqrt((17863792394.913902 / (299)) + (14388643531.135616 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.09430649325162542 = (10741161.804212697 - 9728200.501009095) / 10741161.804212697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.29664355095541 < 103.16668682844067.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 50.83402752160837 (T) = (0 -98557.74512529584) / Math.Sqrt((2110613.0743243136 / (299)) + (1211172.17363209 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1009863378443845 = (109628.75123496834 - 98557.74512529584) / 109628.75123496834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 119.88885865139949 < 122.93300666867974.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 28.496795074905574 (T) = (0 -120398.05952043152) / Math.Sqrt((4178089.756367776 / (299)) + (4695709.503479996 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08903053263428948 = (132164.75835199148 - 120398.05952043152) / 132164.75835199148 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json40KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 996.0681122395832 < 1.0259383241197917.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.351442976421936 (T) = (0 -990895.5068827647) / Math.Sqrt((188170143.57893056 / (299)) + (215063659.47308382 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09060019010384956 = (1089614.8163874377 - 990895.5068827647) / 1089614.8163874377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.69915889261746 < 108.63499696418069.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 29.552223271822342 (T) = (0 -105481.49641141314) / Math.Sqrt((2002612.884280329 / (299)) + (3432656.7465675967 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08884133741116922 = (115766.33219039338 - 105481.49641141314) / 115766.33219039338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.504227212820513 < 9.979117898913044.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 39.44884945097724 (T) = (0 -9620111.609801373) / Math.Sqrt((22972533090.65539 / (299)) + (17580256708.804596 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0954927822142664 = (10635748.859308999 - 9620111.609801373) / 10635748.859308999 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.366320898765434 < 9.735819168958333.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.41125398136155 (T) = (0 -9432674.709576255) / Math.Sqrt((18122270257.939762 / (299)) + (15456714633.766584 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.09405347227628387 = (10411955.254441805 - 9432674.709576255) / 10411955.254441805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.802840254285712 < 10.26786182826087.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 40.9995397333003 (T) = (0 -9960579.754379729) / Math.Sqrt((19537098335.20848 / (299)) + (16957867561.08357 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09368142897129789 = (10990152.99120941 - 9960579.754379729) / 10990152.99120941 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromReader - Duration of single invocation 11.47 μs 10.48 μs 0.91 0.01 False
DeserializeFromString - Duration of single invocation 9.95 μs 9.21 μs 0.93 0.00 False
DeserializeFromUtf8Bytes - Duration of single invocation 7.02 μs 6.29 μs 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.476580251677852 < 10.849552612116451.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 26.333916012308258 (T) = (0 -10585.821549451173) / Math.Sqrt((98892.88454765278 / (119)) + (7539.889992882518 / (30))) is greater than 1.976233308895103 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (119) + (30) - 2, .975) and 0.07565076016048486 = (11452.188299834677 - 10585.821549451173) / 11452.188299834677 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;.DeserializeFromString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.210787192178882 < 9.249533243646418.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.91979405181324 (T) = (0 -9198.007608671536) / Math.Sqrt((15035.567397588733 / (299)) + (6968.819663898008 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06327239813221855 = (9819.298150637638 - 9198.007608671536) / 9819.298150637638 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.286226472453645 < 6.608147502811715.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.5005625624188 (T) = (0 -6329.487885595975) / Math.Sqrt((7316.157685550058 / (299)) + (7000.743974815472 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09023318729956716 = (6957.263990327753 - 6329.487885595975) / 6957.263990327753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a7d80c177a7ea412fb00867ccc4f4b2c44c9e030
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseInt32 - Duration of single invocation 218.09 ns 201.73 ns 0.92 0.11 False
TryParseUInt64Hex - Duration of single invocation 124.73 ns 115.73 ns 0.93 0.25 False
TryParseInt32 - Duration of single invocation 215.95 ns 191.74 ns 0.89 0.08 False
TryParseUInt32Hex - Duration of single invocation 188.30 ns 168.66 ns 0.90 0.20 False
TryParseUInt64Hex - Duration of single invocation 333.75 ns 289.76 ns 0.87 0.08 False
TryParseDouble - Duration of single invocation 1.09 μs 1.01 μs 0.92 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: -2147483648)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 201.72989648713082 < 205.87125554445063.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 6.516846362049494 (T) = (0 -196.44330256443058) / Math.Sqrt((133.9032410651809 / (299)) + (69.30906522011152 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.05154391975932353 = (207.11902918539138 - 196.44330256443058) / 207.11902918539138 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 3039)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.72733679906075 < 118.1810904139228.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 10.1963413254831 (T) = (0 -116.0730584687452) / Math.Sqrt((21.312399520734964 / (299)) + (26.45424868686255 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.08134449830615818 = (126.35101869495863 - 116.0730584687452) / 126.35101869495863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 191.7360875240006 < 202.7830979069618.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 11.84299851582544 (T) = (0 -187.63993038123812) / Math.Sqrt((134.18364725132346 / (299)) + (26.46364797671221 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06934894782058712 = (201.62221913553964 - 187.63993038123812) / 201.62221913553964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.65642123453634 < 178.19600648493304.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 13.002645548376671 (T) = (0 -170.94345609774106) / Math.Sqrt((25.56835544317372 / (299)) + (53.34516089917039 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09691892513849132 = (189.28915781338455 - 170.94345609774106) / 189.28915781338455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 289.7597246503496 < 316.9859069183157.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 39.11080997614283 (T) = (0 -293.6712691322952) / Math.Sqrt((49.24624867520714 / (299)) + (30.142568077735216 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.1258905181271875 = (335.9662321739074 - 293.6712691322952) / 335.9662321739074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0077697393727307 < 1.0376677312481584.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.26988309408164 (T) = (0 -1026.0261004663269) / Math.Sqrt((310.33593623116974 / (299)) + (207.37126769003754 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.05429069120646444 = (1084.9275680444061 - 1026.0261004663269) / 1084.9275680444061 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 1.31 μs 1.20 μs 0.92 0.02 False
DnsSafeHost - Duration of single invocation 9.58 μs 9.01 μs 0.94 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Uri.Ctor

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2010962418237037 < 1.2446514207834702.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 36.37172256605701 (T) = (0 -1215.0171260789764) / Math.Sqrt((316.54905573508864 / (299)) + (247.52698358746346 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.08607669980330411 = (1329.451963657649 - 1215.0171260789764) / 1329.451963657649 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Uri.DnsSafeHost

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.009305916982843 < 9.085607637851934.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 50.55980036648205 (T) = (0 -8967.152015020007) / Math.Sqrt((7112.05012921645 / (299)) + (4665.640533414323 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07116826283699572 = (9654.226547435821 - 8967.152015020007) / 9654.226547435821 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 237.99 ms 224.04 ms 0.94 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]).){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 224.041548 < 224.75395775192305.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 15.909493463134739 (T) = (0 -220822468.51578248) / Math.Sqrt((46225559676412.69 / (299)) + (37289010625882.555 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07958692451800074 = (239916700.8792686 - 220822468.51578248) / 239916700.8792686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_ToLowerInvariant - Duration of single invocation 454.76 ns 379.92 ns 0.84 0.12 False
GetUnicodeCategory - Duration of single invocation 30.32 ns 27.10 ns 0.89 0.21 False
Char_IsUpper - Duration of single invocation 1.25 μs 978.13 ns 0.78 0.14 False
Char_ToUpperInvariant - Duration of single invocation 482.55 ns 415.24 ns 0.86 0.11 False
Char_ToUpper - Duration of single invocation 58.86 ns 53.02 ns 0.90 0.12 False
Char_ToLower - Duration of single invocation 62.06 ns 54.43 ns 0.88 0.14 False
Char_IsLower - Duration of single invocation 1.17 μs 1.02 μs 0.88 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 379.91580293225763 < 418.17619271664546.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 47.083046195458564 (T) = (0 -378.3524386925629) / Math.Sqrt((153.3438062250584 / (299)) + (32.076728968325284 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1381226954164746 = (438.9864272796806 - 378.3524386925629) / 438.9864272796806 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.GetUnicodeCategory(c: 'a')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.10026730032937 < 28.811856004787344.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 7.724340338543077 (T) = (0 -27.77673300839309) / Math.Sqrt((1.7621860864918157 / (299)) + (3.374718360307455 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08732586041817396 = (30.434447305716347 - 27.77673300839309) / 30.434447305716347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 978.1319445331969 < 1.0092044509422693.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 9.687905989177615 (T) = (0 -995.0279201451165) / Math.Sqrt((887.7636969342668 / (299)) + (1212.9589162618342 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06117783522564866 = (1059.8683728182689 - 995.0279201451165) / 1059.8683728182689 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_ToUpperInvariant(input: "Hello World!")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 415.2437967393231 < 444.07738408930555.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 28.312181485057636 (T) = (0 -409.19893767856195) / Math.Sqrt((163.50962863816156 / (299)) + (98.53635649687956 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1208157470712433 = (465.4302398108588 - 409.19893767856195) / 465.4302398108588 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_ToUpper(c: 'a', cultureName: en-US)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 53.02291841586907 < 56.4673598902356.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 34.48414557872452 (T) = (0 -53.69399454529202) / Math.Sqrt((3.9845398181615144 / (299)) + (0.8672589528394449 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11659717472663 = (60.7808725636307 - 53.69399454529202) / 60.7808725636307 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_ToLower(c: 'A', cultureName: en-US)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.431772837597215 < 57.513431944343104.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.089107062056208 (T) = (0 -54.11416267309825) / Math.Sqrt((3.403439898552596 / (299)) + (2.5010578150759586 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10393722767682066 = (60.391039941095904 - 54.11416267309825) / 60.391039941095904 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_IsLower(input: "Good afternoon, Constable!")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0226924710767658 < 1.0377828736456198.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 16.064365119816376 (T) = (0 -1034.5452018209523) / Math.Sqrt((919.735276893205 / (299)) + (567.1683542406467 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.06697717474594682 = (1108.8101746484665 - 1034.5452018209523) / 1108.8101746484665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCredential_Uri - Duration of single invocation 2.33 μs 2.18 μs 0.93 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.17684068994036 < 2.220890911351425.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 23.987461931688458 (T) = (0 -2183.8891872505133) / Math.Sqrt((1031.8663204565419 / (299)) + (1016.0384450324615 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0628801911121227 = (2330.4268744913566 - 2183.8891872505133) / 2330.4268744913566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Exists - Duration of single invocation 3.88 μs 3.48 μs 0.90 0.02 False
ReadAllLines - Duration of single invocation 132.42 μs 120.33 μs 0.91 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_File.Exists

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4770765369005066 < 3.6402683805426888.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 26.274353478406567 (T) = (0 -3523.76169882765) / Math.Sqrt((7434.431750135893 / (299)) + (1911.8642798227372 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.06557946979546021 = (3771.0662222461197 - 3523.76169882765) / 3771.0662222461197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllLines

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 120.33200084935896 < 125.78863025089538.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 102.85675368182514 (T) = (0 -120683.79448829511) / Math.Sqrt((1433964.42094774 / (299)) + (267222.24443502125 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09071822458834057 = (132724.3080767322 - 120683.79448829511) / 132724.3080767322 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 4, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 342.31 μs 296.06 μs 0.86 0.04 False
Parse - Duration of single invocation 14.15 ms 12.77 ms 0.90 0.01 False
Parse - Duration of single invocation 280.19 μs 255.63 μs 0.91 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: NumericProperties)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 296.0595881907009 < 321.29294604974035.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 35.91975337508127 (T) = (0 -299105.9094709017) / Math.Sqrt((37449683.61563999 / (299)) + (21967943.77390199 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1027749536494353 = (333367.76618921396 - 299105.9094709017) / 333367.76618921396 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: ObjectProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.768812213541667 < 13.423760833891402.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 40.178184468474 (T) = (0 -12785121.73222851) / Math.Sqrt((41085091957.79783 / (299)) + (34640972571.902695 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10288953165710643 = (14251446.375210261 - 12785121.73222851) / 14251446.375210261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: StringProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 255.62997579365077 < 265.3517043459596.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 23.22601201031361 (T) = (0 -252132.4580595487) / Math.Sqrt((29558069.893681545 / (299)) + (23475442.72100738 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08070978647301136 = (274268.6197998414 - 252132.4580595487) / 274268.6197998414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 319.24 ns 278.37 ns 0.87 0.15 False
IndexOfString - Duration of single invocation 4.05 μs 3.32 μs 0.82 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 278.3699108436041 < 303.66830734342517.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 45.00199498342589 (T) = (0 -280.3656836711071) / Math.Sqrt((71.43602465301674 / (299)) + (23.79486742469562 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.14345560319560777 = (327.3218349417722 - 280.3656836711071) / 327.3218349417722 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.319476949708455 < 3.8490952148326554.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 41.89558280211325 (T) = (0 -3354.8992572030647) / Math.Sqrt((22889.877108698434 / (299)) + (6460.9717216308245 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.17584497374380814 = (4070.7138224261475 - 3354.8992572030647) / 4070.7138224261475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 11.68 μs 10.20 μs 0.87 0.09 False
IndexOfAnyFiveValues - Duration of single invocation 14.16 μs 12.24 μs 0.86 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.19857768615284 < 11.082315926159223.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 22.822873831534327 (T) = (0 -10259.87094919815) / Math.Sqrt((395543.8811343382 / (299)) + (148258.9446614385 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.15142628844658332 = (12090.724482162212 - 10259.87094919815) / 12090.724482162212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFiveValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.242854241889063 < 13.436570262072054.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 53.09944729047929 (T) = (0 -12248.283503407354) / Math.Sqrt((81400.27892330554 / (90)) + (14241.601978508772 / (28))) is greater than 1.9806260024587088 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (90) + (28) - 2, .975) and 0.14012978753036756 = (14244.33981522522 - 12248.283503407354) / 14244.33981522522 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 1.92 μs 1.62 μs 0.84 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address: 1020:3040:5060:7080:9010:1112:1314:1516)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6160419252102343 < 1.8080962290777987.
IsChangePoint: Marked as a change because one of 8/5/2022 8:01:14 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 19.517560946837914 (T) = (0 -1643.9498114015155) / Math.Sqrt((8072.854390110984 / (299)) + (894.7769954997586 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08214521414237967 = (1791.0783238608385 - 1643.9498114015155) / 1791.0783238608385 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractJsonSerializer_ - Duration of single invocation 1.78 ms 1.66 ms 0.93 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.DataContractJsonSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.656624354777673 < 1.6804612728169535.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 55.85554276065484 (T) = (0 -1649992.157398556) / Math.Sqrt((326411964.6925333 / (299)) + (141988198.26392522 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.0755286311691108 = (1784795.3035962372 - 1649992.157398556) / 1784795.3035962372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 7.37 ms 6.91 ms 0.94 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromStream

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.909431872685186 < 6.969441120233932.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 37.94134573194718 (T) = (0 -6947051.954262566) / Math.Sqrt((52503689418.45515 / (299)) + (4198076048.5627713 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08904835153124802 = (7626147.84872511 - 6947051.954262566) / 7626147.84872511 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BenchAssignJagged - Duration of single invocation 8.56 secs 6.72 secs 0.78 0.02 False
BenchEmFloatClass - Duration of single invocation 4.96 secs 4.56 secs 0.92 0.01 False
BenchIDEAEncryption - Duration of single invocation 10.63 secs 9.05 secs 0.85 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

ByteMark.BenchAssignJagged

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.721189723399999 < 8.13660494732077.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 184.12392622572273 (T) = (0 -6676161486.064483) / Math.Sqrt((8697149972362482 / (299)) + (2421909073986320 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.22869586392083624 = (8655679612.975998 - 6676161486.064483) / 8655679612.975998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchEmFloatClass

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.560235351428572 < 4.716679759192307.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 43.79316705752996 (T) = (0 -4590648034.822565) / Math.Sqrt((3488844083845764.5 / (299)) + (1951406023044610 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07814375605440714 = (4979787320.389947 - 4590648034.822565) / 4979787320.389947 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchIDEAEncryption

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.049156487642856 < 10.068458862784999.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 56.02382981780213 (T) = (0 -9071492927.847738) / Math.Sqrt((36682433095013450 / (299)) + (16739320420281244 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.13717166078572665 = (10513670582.619724 - 9071492927.847738) / 10513670582.619724 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 164.33 ns 122.70 ns 0.75 0.12 False
InequalityOperatorBenchmark - Duration of single invocation 23.14 ns 21.54 ns 0.93 0.21 False
EqualsAllBenchmark - Duration of single invocation 22.71 ns 20.67 ns 0.91 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt16>.DotBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 122.69686682251773 < 144.93819846786994.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 56.424060645628124 (T) = (0 -123.7443475168066) / Math.Sqrt((31.805757399562953 / (299)) + (5.807096156709291 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.20345002501413462 = (155.35038780084378 - 123.7443475168066) / 155.35038780084378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.541916119514944 < 21.9949741795478.
IsChangePoint: Marked as a change because one of 9/30/2022 8:27:04 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 20.9709895961875 (T) = (0 -21.413232718231956) / Math.Sqrt((4.338710858187341 / (299)) + (0.047421419063740824 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.11254709959090302 = (24.128866679415786 - 21.413232718231956) / 24.128866679415786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.668366687489616 < 21.54667935720588.
IsChangePoint: Marked as a change because one of 6/23/2022 10:44:43 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 4.863480289982432 (T) = (0 -21.507770544140865) / Math.Sqrt((1.9829819019193065 / (299)) + (3.565386123277491 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07527124118593645 = (23.25846399729573 - 21.507770544140865) / 23.25846399729573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDouble - Duration of single invocation 2.20 μs 1.90 μs 0.86 0.25 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9028843519082423 < 2.109447313887603.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 17.922753640336282 (T) = (0 -1983.6735409846665) / Math.Sqrt((4709.712638084983 / (299)) + (3026.990648780959 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09136629861474575 = (2183.138857782255 - 1983.6735409846665) / 2183.138857782255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline af88ee90bacedb49b90f7dc62a3092b287fc6ed0
Compare 789b420913a93f243adb4a6b14b162dc99c01242
Diff Diff

Improvements in System.IO.Tests.Perf_FileInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ctor_str - Duration of single invocation 2.14 μs 2.03 μs 0.95 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileInfo*'

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.Perf_FileInfo.ctor_str

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.031657013725974 < 2.090722792750181.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsImprovementStdDev: Marked as improvement because 38.833471429977315 (T) = (0 -1905.8591799203966) / Math.Sqrt((37490.09793055792 / (299)) + (2098.458272366885 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2227148446409439 = (2451.9433656751253 - 1905.8591799203966) / 2451.9433656751253 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@kotlarmilos
Copy link
Member

Looks like regression in dotnet/runtime@fbb3d57

@kotlarmilos kotlarmilos mentioned this issue Oct 26, 2022
Closed
@kotlarmilos
Copy link
Member

According to the current data, not a regression but noise, also a lot of improvements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement perf-regression runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos