Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 88 Regressions on 11/2/2022 4:25:40 PM #9637

Open
performanceautofiler bot opened this issue Nov 8, 2022 · 8 comments
Open

[Perf] Linux/x64: 88 Regressions on 11/2/2022 4:25:40 PM #9637

performanceautofiler bot opened this issue Nov 8, 2022 · 8 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged wasm-aot

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndBenchmark - Duration of single invocation 30.63 ns 75.35 ns 2.46 0.52 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 19.96 ns 76.92 ns 3.85 0.53 False
BitwiseOrOperatorBenchmark - Duration of single invocation 20.86 ns 75.56 ns 3.62 0.50 False
ConditionalSelectBenchmark - Duration of single invocation 212.54 ns 265.27 ns 1.25 0.26 False
InequalityOperatorBenchmark - Duration of single invocation 55.52 ns 194.64 ns 3.51 0.55 False
XorBenchmark - Duration of single invocation 34.32 ns 75.83 ns 2.21 0.52 False
CountBenchmark - Duration of single invocation 11.48 ns 20.23 ns 1.76 0.61 False
BitwiseOrBenchmark - Duration of single invocation 32.41 ns 72.73 ns 2.24 0.53 False
BitwiseAndOperatorBenchmark - Duration of single invocation 20.03 ns 76.54 ns 3.82 0.50 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt16>.BitwiseAndBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.35409787114281 > 32.17841036200589.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -94.34656723652652 (T) = (0 -75.49217186345241) / Math.Sqrt((2.396234331306331 / (22)) + (1.4780869759584763 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -1.3742190750053471 = (31.79663269417646 - 75.49217186345241) / 31.79663269417646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.92404876807004 > 22.920030896745295.
IsChangePoint: Marked as a change because one of 11/2/2022 4:14:20 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -95.28976956604316 (T) = (0 -76.18216823529201) / Math.Sqrt((4.198962514735415 / (22)) + (1.8593741233546248 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.467626322310665 = (21.96954376114199 - 76.18216823529201) / 21.96954376114199 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.55586244228529 > 21.854414632045344.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -84.58618541252041 (T) = (0 -75.98360305046292) / Math.Sqrt((1.7671198861386077 / (22)) + (4.796675682299736 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.6227962174501616 = (20.973744723610917 - 75.98360305046292) / 20.973744723610917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 265.26762617699393 > 192.90756131016354.
IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -37.03001723466712 (T) = (0 -257.48905145801297) / Math.Sqrt((66.8931694190453 / (23)) + (5.862061265976367 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.35559890449255205 = (189.94486540574485 - 257.48905145801297) / 189.94486540574485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 194.63629146611163 > 61.40772508366534.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -33.50818430171248 (T) = (0 -204.99032874644416) / Math.Sqrt((10.485134865554752 / (22)) + (237.25476572038897 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -2.4173715513844516 = (59.984794062968696 - 204.99032874644416) / 59.984794062968696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.8258413852479 > 36.0062093841828.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -70.29503614534421 (T) = (0 -76.22324844679663) / Math.Sqrt((3.17837267809177 / (21)) + (3.123187545356091 / (14))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (14) - 2, .025) and -1.2952660425501439 = (33.20889475718866 - 76.22324844679663) / 33.20889475718866 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.CountBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.227336105410796 > 11.974643666674496.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -23.646810078758815 (T) = (0 -20.709067860481703) / Math.Sqrt((2.478769471590619 / (23)) + (0.4157186001030311 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.7341873356889589 = (11.941655572207482 - 20.709067860481703) / 11.941655572207482 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 72.73430911408717 > 36.671311735835644.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -41.31639115297557 (T) = (0 -77.69737475314955) / Math.Sqrt((6.585242866016997 / (22)) + (10.328078483215693 / (12))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (12) - 2, .025) and -1.340408340942722 = (33.1982130613382 - 77.69737475314955) / 33.1982130613382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.53702655779067 > 21.20034247123962.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -39.77531416521281 (T) = (0 -76.99425613009747) / Math.Sqrt((0.0916314189228407 / (23)) + (28.296527152123726 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.775950916826518 = (20.390693053501597 - 76.99425613009747) / 20.390693053501597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExclusiveOrOperatorBenchmark - Duration of single invocation 20.30 ns 75.53 ns 3.72 0.53 False
InequalityOperatorBenchmark - Duration of single invocation 58.13 ns 353.96 ns 6.09 0.52 False
XorBenchmark - Duration of single invocation 34.99 ns 75.00 ns 2.14 0.48 False
BitwiseAndBenchmark - Duration of single invocation 30.52 ns 75.38 ns 2.47 0.53 False
ZeroBenchmark - Duration of single invocation 16.39 ns 22.84 ns 1.39 0.56 False
BitwiseOrOperatorBenchmark - Duration of single invocation 20.25 ns 75.26 ns 3.72 0.51 False
BitwiseOrBenchmark - Duration of single invocation 30.91 ns 76.53 ns 2.48 0.49 False
BitwiseAndOperatorBenchmark - Duration of single invocation 20.25 ns 75.44 ns 3.73 0.54 False
CountBenchmark - Duration of single invocation 12.54 ns 20.52 ns 1.64 0.66 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.5278293603989 > 21.524064127247424.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -126.8095839291934 (T) = (0 -76.07474391151) / Math.Sqrt((2.4873420055967057 / (22)) + (0.9829909411297725 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -2.6238666148874885 = (20.992699786184573 - 76.07474391151) / 20.992699786184573 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 353.96203728452457 > 61.23097520390926.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -71.37762213928872 (T) = (0 -360.6496930690451) / Math.Sqrt((7.560166275591441 / (23)) + (227.1738490879186 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -5.063723807601269 = (59.47660291139043 - 360.6496930690451) / 59.47660291139043 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.00265277437725 > 35.07568994773114.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -86.44707492131504 (T) = (0 -75.44209849841856) / Math.Sqrt((4.548838260515828 / (23)) + (0.48155979241609526 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.2485463456120922 = (33.551498124839384 - 75.44209849841856) / 33.551498124839384 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.38498050342236 > 32.08272472948031.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -65.26577151345143 (T) = (0 -76.26457103536988) / Math.Sqrt((5.496326937083195 / (22)) + (2.4517160818366763 / (12))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (12) - 2, .025) and -1.3624564724676684 = (32.28189468215212 - 76.26457103536988) / 32.28189468215212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.844032585360146 > 18.270089489510756.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -3.8241970127852647 (T) = (0 -23.11459442568861) / Math.Sqrt((22.878560061569285 / (23)) + (0.799105378728803 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.20486247319039985 = (19.184425558945843 - 23.11459442568861) / 19.184425558945843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.2645867367288 > 21.715156500731666.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -84.83832567908212 (T) = (0 -76.2582679105188) / Math.Sqrt((3.4525190981059173 / (23)) + (3.7248644360780583 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.542294661307101 = (21.527928984422662 - 76.2582679105188) / 21.527928984422662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.52860213226606 > 32.500880161390434.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -65.89651427323399 (T) = (0 -75.4846146401836) / Math.Sqrt((8.488197233903877 / (23)) + (0.5998858251541921 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.2741432825155214 = (33.192550012366425 - 75.4846146401836) / 33.192550012366425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.44373177392644 > 21.36212394584959.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -84.06500751545441 (T) = (0 -75.91686670910981) / Math.Sqrt((6.950357472157528 / (22)) + (1.2818986413722038 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.411373401068801 = (22.254047793573303 - 75.91686670910981) / 22.254047793573303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.CountBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.522979556396784 > 13.238654540930112.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -18.174745150624354 (T) = (0 -20.82689317453668) / Math.Sqrt((3.329466979566583 / (23)) + (0.8297845751531917 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.6506454561170153 = (12.61742374618105 - 20.82689317453668) / 12.61742374618105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseOrBenchmark - Duration of single invocation 30.54 ns 80.30 ns 2.63 0.56 False
BitwiseOrOperatorBenchmark - Duration of single invocation 20.37 ns 75.41 ns 3.70 0.55 False
ConditionalSelectBenchmark - Duration of single invocation 152.19 ns 259.10 ns 1.70 0.33 False
XorBenchmark - Duration of single invocation 32.24 ns 76.38 ns 2.37 0.52 False
BitwiseAndBenchmark - Duration of single invocation 30.93 ns 75.21 ns 2.43 0.57 False
ZeroBenchmark - Duration of single invocation 16.44 ns 22.53 ns 1.37 0.58 False
BitwiseAndOperatorBenchmark - Duration of single invocation 20.34 ns 75.54 ns 3.71 0.56 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 23.68 ns 80.96 ns 3.42 0.55 False
CountBenchmark - Duration of single invocation 13.06 ns 20.55 ns 1.57 0.60 False
InequalityOperatorBenchmark - Duration of single invocation 59.69 ns 102.65 ns 1.72 0.44 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int32>.BitwiseOrBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 80.29830110958977 > 34.08604328709482.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -46.44781167480066 (T) = (0 -76.70879609412285) / Math.Sqrt((4.480512306401616 / (23)) + (9.112441577286178 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.3423674665745458 = (32.74840399243634 - 76.70879609412285) / 32.74840399243634 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.40666012484307 > 21.359133469814385.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -80.47406697627052 (T) = (0 -75.62872435721358) / Math.Sqrt((1.512878929629931 / (23)) + (5.56563190652298 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.626963677413825 = (20.85180086808589 - 75.62872435721358) / 20.85180086808589 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 259.10354993597946 > 165.18577819570507.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -51.44114411994325 (T) = (0 -259.3837494869896) / Math.Sqrt((43.21035071606401 / (22)) + (24.745536859334177 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -0.6394035574516672 = (158.2183644216209 - 259.3837494869896) / 158.2183644216209 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.38386050057645 > 33.48973459594621.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -105.46153382491384 (T) = (0 -75.78968479456269) / Math.Sqrt((3.0719520082511007 / (23)) + (0.5395222277836725 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.3655462007952244 = (32.03897889167606 - 75.78968479456269) / 32.03897889167606 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.20807114497033 > 32.28512558296767.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -66.24018994788575 (T) = (0 -76.30964975372333) / Math.Sqrt((4.781587527388794 / (22)) + (3.04643650532458 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -1.4003382000269278 = (31.791207486039788 - 76.30964975372333) / 31.791207486039788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.533013702457534 > 17.40944963945943.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -10.939637042695233 (T) = (0 -22.376684387469105) / Math.Sqrt((1.6559855167601036 / (23)) + (2.1763391196157373 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.3040513233839667 = (17.159358674168136 - 22.376684387469105) / 17.159358674168136 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.53600585121836 > 21.30681675711573.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -33.59049123439223 (T) = (0 -78.51145245188526) / Math.Sqrt((3.3880721042680344 / (22)) + (38.44868364059195 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.685019104427474 = (21.305575419556273 - 78.51145245188526) / 21.305575419556273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 80.95593145661492 > 21.27129950888574.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -95.9408251096406 (T) = (0 -76.30212503115865) / Math.Sqrt((2.167390737772101 / (23)) + (3.339467106517318 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.6407720759760434 = (20.957676953920014 - 76.30212503115865) / 20.957676953920014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.CountBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.5549466267448 > 13.526549930605368.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -16.83078291070081 (T) = (0 -21.04165080690312) / Math.Sqrt((2.7867174915510264 / (23)) + (1.7859131791765155 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.6855247838977006 = (12.483738600535583 - 21.04165080690312) / 12.483738600535583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 102.65233487265782 > 65.13385631242198.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -16.232562833998887 (T) = (0 -107.77789381787804) / Math.Sqrt((11.36703171379025 / (23)) + (101.33695919085544 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.7656265952082061 = (61.04229179056326 - 107.77789381787804) / 61.04229179056326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConditionalSelectBenchmark - Duration of single invocation 136.72 ns 259.11 ns 1.90 0.40 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 20.27 ns 75.03 ns 3.70 0.48 False
BitwiseOrOperatorBenchmark - Duration of single invocation 21.69 ns 80.78 ns 3.72 0.59 False
AndNotBenchmark - Duration of single invocation 106.59 ns 123.76 ns 1.16 0.40 False
BitwiseAndBenchmark - Duration of single invocation 32.96 ns 79.93 ns 2.42 0.51 False
XorBenchmark - Duration of single invocation 31.20 ns 75.25 ns 2.41 0.56 False
BitwiseAndOperatorBenchmark - Duration of single invocation 24.17 ns 75.25 ns 3.11 0.54 False
CountBenchmark - Duration of single invocation 11.38 ns 22.56 ns 1.98 0.71 False
BitwiseOrBenchmark - Duration of single invocation 30.00 ns 75.24 ns 2.51 0.55 False
ZeroBenchmark - Duration of single invocation 15.14 ns 22.32 ns 1.47 0.65 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Double>.ConditionalSelectBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 259.10951540984723 > 158.71683556976728.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -24.70948954788735 (T) = (0 -265.58016521561683) / Math.Sqrt((75.10203695425574 / (22)) + (261.89534465825915 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -0.8234777447927895 = (145.64486239221745 - 265.58016521561683) / 145.64486239221745 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.03318115455872 > 22.33774272763171.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -91.09543336925968 (T) = (0 -75.77194991364188) / Math.Sqrt((2.726752061274602 / (22)) + (3.217079364770818 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.509195772018683 = (21.592397471188583 - 75.77194991364188) / 21.592397471188583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 80.78332228628508 > 22.925894776708603.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -72.96902692626242 (T) = (0 -76.31837881475933) / Math.Sqrt((8.383134244377684 / (23)) + (2.465635432007257 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -2.469111531989591 = (21.999401896136074 - 76.31837881475933) / 21.999401896136074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.AndNotBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 123.75540901071345 > 116.18092988994057.
IsChangePoint: Marked as a change because one of 9/24/2022 7:43:48 AM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -10.597843218574878 (T) = (0 -124.56288381070624) / Math.Sqrt((32.2303727693615 / (23)) + (1.2276956991052603 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.11584639121495482 = (111.63085241067975 - 124.56288381070624) / 111.63085241067975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 79.92683067362333 > 34.922805671421045.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -66.61326726704661 (T) = (0 -75.72705856940884) / Math.Sqrt((6.378183879893301 / (22)) + (1.5285347032030823 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -1.2507917596375993 = (33.64463115930444 - 75.72705856940884) / 33.64463115930444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.24585397253556 > 33.42264336596443.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -71.86966218214553 (T) = (0 -76.01436562681603) / Math.Sqrt((4.819837021914824 / (23)) + (2.069476686065023 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.299987984581723 = (33.04989684136982 - 76.01436562681603) / 33.04989684136982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.25126654613773 > 23.815523949404273.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -100.11494195304009 (T) = (0 -76.02121703651402) / Math.Sqrt((3.31493277539242 / (23)) + (2.0892457339574375 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.4878031798580476 = (21.796303608969144 - 76.02121703651402) / 21.796303608969144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.CountBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.559895342529735 > 12.643312959360095.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -17.09606051117243 (T) = (0 -20.799731666679428) / Math.Sqrt((1.9180379059301196 / (23)) + (2.3219111306457374 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.7262723723872759 = (12.048928083066818 - 20.799731666679428) / 12.048928083066818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.244061146002 > 32.39373139893763.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -69.31830436981475 (T) = (0 -75.55918488526697) / Math.Sqrt((8.164963366966548 / (23)) + (0.4264338673200779 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.323272822883222 = (32.5227343689652 - 75.55918488526697) / 32.5227343689652 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.320348532974695 > 18.35368027989347.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -9.963969078746777 (T) = (0 -22.539163349953352) / Math.Sqrt((3.5046368364092806 / (22)) + (1.4338283153475186 / (12))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (12) - 2, .025) and -0.3044889440434355 = (17.27815590379037 - 22.539163349953352) / 17.27815590379037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CountBenchmark - Duration of single invocation 13.17 ns 20.67 ns 1.57 0.65 False
BitwiseAndOperatorBenchmark - Duration of single invocation 26.92 ns 75.16 ns 2.79 0.54 False
InequalityOperatorBenchmark - Duration of single invocation 56.29 ns 101.78 ns 1.81 0.49 False
BitwiseAndBenchmark - Duration of single invocation 31.37 ns 76.70 ns 2.44 0.58 False
BitwiseOrOperatorBenchmark - Duration of single invocation 20.24 ns 74.86 ns 3.70 0.53 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 20.25 ns 75.28 ns 3.72 0.54 False
ConditionalSelectBenchmark - Duration of single invocation 158.08 ns 256.51 ns 1.62 0.32 False
ZeroBenchmark - Duration of single invocation 20.00 ns 22.59 ns 1.13 0.60 False
BitwiseOrBenchmark - Duration of single invocation 30.52 ns 75.44 ns 2.47 0.57 False
XorBenchmark - Duration of single invocation 31.20 ns 73.34 ns 2.35 0.52 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt32>.CountBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.671157007527086 > 12.367207000882422.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -20.001739643183843 (T) = (0 -21.393411459558912) / Math.Sqrt((0.9849554209792596 / (23)) + (2.3561273350470877 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.7939225853167058 = (11.92549312588204 - 21.393411459558912) / 11.92549312588204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.15514994177903 > 25.88502746203165.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -85.68704322965515 (T) = (0 -75.5830001522019) / Math.Sqrt((7.8973659898629345 / (23)) + (0.45335971695233873 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.2778929819169167 = (23.058409950894998 - 75.5830001522019) / 23.058409950894998 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.77726756385083 > 61.45541528504369.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -41.740991907681355 (T) = (0 -104.25233342093996) / Math.Sqrt((11.355006018298456 / (22)) + (8.858929435029413 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.7518108202502333 = (59.511182495178495 - 104.25233342093996) / 59.511182495178495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.70441245872834 > 34.20209623713422.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -55.255685327632 (T) = (0 -76.38561065411902) / Math.Sqrt((6.380952947002364 / (23)) + (4.937731839468723 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.3486412775271421 = (32.52331949753713 - 76.38561065411902) / 32.52331949753713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 74.85792403417051 > 21.649663349824028.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -47.577214157927685 (T) = (0 -76.7826484285955) / Math.Sqrt((1.9628686541091078 / (23)) + (17.83356918924662 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.602193919485968 = (21.315523301852764 - 76.7826484285955) / 21.315523301852764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.28460883189487 > 21.36683840849005.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -176.9350082285564 (T) = (0 -75.4742576464175) / Math.Sqrt((1.0964860201636066 / (23)) + (0.6258348089938666 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -2.6450503835134587 = (20.70595731345364 - 75.4742576464175) / 20.70595731345364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 256.50890992719684 > 167.62314650795537.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -64.49771638734302 (T) = (0 -259.97309714858943) / Math.Sqrt((21.483937843343874 / (23)) + (22.080404188333464 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.6478613948200869 = (157.76393449460792 - 259.97309714858943) / 157.76393449460792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.58913099672888 > 17.393391486327268.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -13.245108119887117 (T) = (0 -23.20217745969098) / Math.Sqrt((2.436888975787334 / (22)) + (0.9759687109049526 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.32016046737963183 = (17.57527060762898 - 23.20217745969098) / 17.57527060762898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.44020517600943 > 34.459602223248794.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -54.14870967235584 (T) = (0 -75.85182847783616) / Math.Sqrt((9.475922585254093 / (22)) + (2.42560092953408 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -1.2461636419478828 = (33.769502391222545 - 75.85182847783616) / 33.769502391222545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 73.33707174806803 > 35.803029212869355.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -61.64599901420579 (T) = (0 -75.4865868098507) / Math.Sqrt((8.5961658825039 / (23)) + (0.8189430744985108 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.1724677487950648 = (34.746930927613775 - 75.4865868098507) / 34.746930927613775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CountBenchmark - Duration of single invocation 11.19 ns 20.56 ns 1.84 0.67 False
ZeroBenchmark - Duration of single invocation 14.44 ns 24.95 ns 1.73 0.58 False
BitwiseAndBenchmark - Duration of single invocation 30.54 ns 76.44 ns 2.50 0.50 False
BitwiseAndOperatorBenchmark - Duration of single invocation 20.04 ns 75.07 ns 3.75 0.49 False
InequalityOperatorBenchmark - Duration of single invocation 55.99 ns 355.80 ns 6.36 0.43 False
BitwiseOrBenchmark - Duration of single invocation 38.14 ns 74.67 ns 1.96 0.55 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 20.75 ns 75.18 ns 3.62 0.46 False
XorBenchmark - Duration of single invocation 31.14 ns 77.38 ns 2.48 0.59 False
BitwiseOrOperatorBenchmark - Duration of single invocation 20.25 ns 75.28 ns 3.72 0.54 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.CountBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.559842847233906 > 11.790653201416884.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -16.45725542304856 (T) = (0 -20.25167980086153) / Math.Sqrt((1.0937470086266958 / (23)) + (2.9330984540366063 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.7383308067448668 = (11.650072427114186 - 20.25167980086153) / 11.650072427114186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.951445390335625 > 17.385548850408362.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 11/2/2022 4:14:20 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -11.009105491038632 (T) = (0 -22.551048866187365) / Math.Sqrt((2.4711416557486046 / (21)) + (1.7746252729040426 / (15))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (15) - 2, .025) and -0.31087499179234096 = (17.203050639751417 - 22.551048866187365) / 17.203050639751417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.44295577870831 > 33.13660243911368.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -56.36791759539824 (T) = (0 -75.86305019954206) / Math.Sqrt((12.069336316653073 / (23)) + (0.4222729865140123 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.245440204749108 = (33.78537982845931 - 75.86305019954206) / 33.78537982845931 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.06910874223036 > 22.018761222075135.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -127.59318764697694 (T) = (0 -75.42650391460575) / Math.Sqrt((3.1601691734164423 / (23)) + (0.5763563237110838 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.5067065682881107 = (21.50921454241526 - 75.42650391460575) / 21.50921454241526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 355.79582344719034 > 60.25133119409198.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -52.146107366317196 (T) = (0 -368.75733526066443) / Math.Sqrt((13.246719654006686 / (22)) + (481.9666952430247 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -5.132460550855075 = (60.13203545341144 - 368.75733526066443) / 60.13203545341144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 74.67044361821645 > 35.53114912890451.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -63.8875846098325 (T) = (0 -75.63282544724355) / Math.Sqrt((5.4061170506926075 / (22)) + (2.6693841567743166 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -1.2625470569248496 = (33.4281778651889 - 75.63282544724355) / 33.4281778651889 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.1811584608038 > 21.29038489796157.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -30.95516756744419 (T) = (0 -78.2674079271803) / Math.Sqrt((1.3972759322111474 / (23)) + (47.63560122162918 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.7883427653023887 = (20.660065040585874 - 78.2674079271803) / 20.660065040585874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 77.38481509869604 > 35.0570775969305.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -56.371601623856016 (T) = (0 -76.67887149971665) / Math.Sqrt((4.338351037381586 / (23)) + (5.675991017593305 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.3884002135420983 = (32.104699649979786 - 76.67887149971665) / 32.104699649979786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.27739592643458 > 22.85519812097911.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -88.92739454826948 (T) = (0 -75.88899945532607) / Math.Sqrt((5.941856441634825 / (23)) + (1.3736445961628438 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -2.4128162385503114 = (22.2364739707058 - 75.88899945532607) / 22.2364739707058 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExclusiveOrOperatorBenchmark - Duration of single invocation 20.25 ns 75.55 ns 3.73 0.56 False
XorBenchmark - Duration of single invocation 31.43 ns 75.36 ns 2.40 0.49 False
BitwiseAndOperatorBenchmark - Duration of single invocation 20.92 ns 75.49 ns 3.61 0.51 False
BitwiseAndBenchmark - Duration of single invocation 31.54 ns 79.91 ns 2.53 0.56 False
CountBenchmark - Duration of single invocation 10.89 ns 20.22 ns 1.86 0.61 False
ZeroBenchmark - Duration of single invocation 16.49 ns 22.58 ns 1.37 0.54 False
BitwiseOrOperatorBenchmark - Duration of single invocation 20.17 ns 76.96 ns 3.82 0.58 False
ConditionalSelectBenchmark - Duration of single invocation 145.24 ns 256.51 ns 1.77 0.32 False
BitwiseOrBenchmark - Duration of single invocation 31.49 ns 76.23 ns 2.42 0.48 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt64>.ExclusiveOrOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.55075415568692 > 21.403697027569503.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -109.88338261335383 (T) = (0 -75.86098974466604) / Math.Sqrt((1.2541090453001804 / (23)) + (2.5427261449790177 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -2.6286661515557825 = (20.90602623009031 - 75.86098974466604) / 20.90602623009031 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.36357104223853 > 33.49665335608814.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -98.13426944381682 (T) = (0 -75.95336624027787) / Math.Sqrt((2.82841068189177 / (22)) + (0.8861267129656323 / (12))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (12) - 2, .025) and -1.3882471299468786 = (31.80297603538512 - 75.95336624027787) / 31.80297603538512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.48871797810901 > 22.262745513016675.
IsChangePoint: Marked as a change because one of 11/2/2022 4:14:20 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -83.246389510088 (T) = (0 -76.10562417479234) / Math.Sqrt((4.474517056803682 / (22)) + (3.0092473048804655 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.4184886596370574 = (22.26294475491181 - 76.10562417479234) / 22.26294475491181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 79.90516809968592 > 33.566740866202686.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -74.5616383975748 (T) = (0 -76.07355056325162) / Math.Sqrt((5.099538657583325 / (23)) + (1.5346415030547256 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.2945156416907844 = (33.15451382462334 - 76.07355056325162) / 33.15451382462334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.CountBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.2233997569551 > 11.811686362008315.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -20.120304112324856 (T) = (0 -21.150791422525515) / Math.Sqrt((1.8154448774577736 / (23)) + (1.6187353379115494 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.7229357253188526 = (12.276018839072638 - 21.150791422525515) / 12.276018839072638 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.57990801097219 > 17.538919729597225.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -15.067797195949174 (T) = (0 -22.626563434896067) / Math.Sqrt((2.5703853055744825 / (23)) + (0.0255782375945467 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.2893800532322117 = (17.548405047973176 - 22.626563434896067) / 17.548405047973176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.96177811757374 > 21.4581562216694.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -76.68538038647941 (T) = (0 -76.16845183983344) / Math.Sqrt((5.389711474607306 / (22)) + (3.3916150385052597 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -2.5221149930903546 = (21.62577087609571 - 76.16845183983344) / 21.62577087609571 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 256.50614431594636 > 153.4195420290781.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -66.65604579710083 (T) = (0 -262.01447356322814) / Math.Sqrt((28.164091236454166 / (23)) + (26.894854379321746 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.8576049193934981 = (141.0496229999084 - 262.01447356322814) / 141.0496229999084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.2262652651422 > 33.830944405397894.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -53.19753385707992 (T) = (0 -76.77925772549808) / Math.Sqrt((7.984040784122808 / (23)) + (4.700338324102728 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.3394794254350662 = (32.81894975897028 - 76.77925772549808) / 32.81894975897028 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseUInt64 - Duration of single invocation 33.46 ns 42.61 ns 1.27 0.52 False
TryParseUInt64Hex - Duration of single invocation 29.12 ns 33.11 ns 1.14 0.61 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 42.61322079813015 > 36.48536920749092.
IsChangePoint: Marked as a change because one of 9/19/2022 6:07:30 AM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -6.064739985695188 (T) = (0 -39.67460751465472) / Math.Sqrt((4.745715461806133 / (23)) + (4.380619063877834 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.12698090431801154 = (35.2043298716438 - 39.67460751465472) / 35.2043298716438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 33.114685742811574 > 32.23139091861525.
IsChangePoint: Marked as a change because one of 9/27/2022 6:39:02 PM, 11/2/2022 4:14:20 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -3.378104757233599 (T) = (0 -36.70558560318974) / Math.Sqrt((5.439123119755085 / (22)) + (18.016148859513315 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.12865581011686572 = (32.52150502763911 - 36.70558560318974) / 32.52150502763911 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndOperatorBenchmark - Duration of single invocation 20.80 ns 72.24 ns 3.47 0.57 False
XorBenchmark - Duration of single invocation 33.14 ns 75.28 ns 2.27 0.53 False
BitwiseOrOperatorBenchmark - Duration of single invocation 20.68 ns 80.69 ns 3.90 0.56 False
ZeroBenchmark - Duration of single invocation 16.67 ns 22.47 ns 1.35 0.58 False
ConditionalSelectBenchmark - Duration of single invocation 183.91 ns 266.05 ns 1.45 0.26 False
BitwiseOrBenchmark - Duration of single invocation 31.47 ns 75.50 ns 2.40 0.55 False
InequalityOperatorBenchmark - Duration of single invocation 57.32 ns 191.07 ns 3.33 0.48 False
CountBenchmark - Duration of single invocation 13.53 ns 22.31 ns 1.65 0.66 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 20.62 ns 75.47 ns 3.66 0.49 False
BitwiseAndBenchmark - Duration of single invocation 32.56 ns 74.91 ns 2.30 0.60 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseAndOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 72.24159813878545 > 21.292412694270475.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -76.1739952247823 (T) = (0 -76.07673913177085) / Math.Sqrt((1.432245088463102 / (22)) + (5.933800708153527 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -2.611493130343269 = (21.065176198892512 - 76.07673913177085) / 21.065176198892512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.28169046869012 > 32.07152634070827.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -75.42059852164975 (T) = (0 -76.25399337345395) / Math.Sqrt((4.316790941042026 / (23)) + (2.2509968910238523 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.4030579146486384 = (31.732066426124142 - 76.25399337345395) / 31.732066426124142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 80.685827786579 > 21.85084873340416.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -81.13256713600308 (T) = (0 -77.11641347062641) / Math.Sqrt((1.4954514462464668 / (23)) + (5.385378534883473 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -2.6812688853743802 = (20.948324034956705 - 77.11641347062641) / 20.948324034956705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.466578498841745 > 18.581608310536073.
IsChangePoint: Marked as a change because one of 8/2/2022 9:12:32 AM, 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -14.387210574008567 (T) = (0 -22.898897599016923) / Math.Sqrt((2.5183954995735194 / (22)) + (0.46331623542781153 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.31813926046421326 = (17.372138351263846 - 22.898897599016923) / 17.372138351263846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 266.0495209682667 > 191.76908686521585.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -41.60336093657482 (T) = (0 -259.7341967606426) / Math.Sqrt((45.976877542318874 / (23)) + (13.586015568260608 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -0.38785685996792024 = (187.14768377961272 - 259.7341967606426) / 187.14768377961272 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.5038997942774 > 33.519117663162255.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -53.983298367081204 (T) = (0 -76.04795794702059) / Math.Sqrt((7.689378667864364 / (23)) + (4.272309871975872 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.3695762887565974 = (32.093483678014735 - 76.04795794702059) / 32.093483678014735 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 191.06650463147125 > 61.22836666059523.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -48.9927386842354 (T) = (0 -200.41938112445717) / Math.Sqrt((5.330837511981421 / (22)) + (114.09662969304993 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.426442400573538 = (58.491974384542345 - 200.41938112445717) / 58.491974384542345 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.CountBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.31456230453325 > 13.551087459064187.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -23.04829292066575 (T) = (0 -21.31486370796701) / Math.Sqrt((1.2196544014922415 / (23)) + (1.5078140397630224 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.7652712224886541 = (12.074554570666836 - 21.31486370796701) / 12.074554570666836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.46843689185648 > 21.292391005835043.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -104.80849085657248 (T) = (0 -75.89537424695101) / Math.Sqrt((3.4325798848034874 / (23)) + (1.596092023857156 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -2.5745660403837 = (21.232052615484555 - 75.89537424695101) / 21.232052615484555 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.BitwiseAndBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 74.9130361070458 > 34.21510441427719.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -54.356432137021216 (T) = (0 -75.93903087003368) / Math.Sqrt((10.59874447766324 / (23)) + (1.8901650003591284 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.2347233578813337 = (33.98140114400063 - 75.93903087003368) / 33.98140114400063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitwiseAndBenchmark - Duration of single invocation 31.85 ns 75.47 ns 2.37 0.51 False
CountBenchmark - Duration of single invocation 14.52 ns 20.55 ns 1.42 0.62 False
ConditionalSelectBenchmark - Duration of single invocation 171.86 ns 254.77 ns 1.48 0.38 False
BitwiseAndOperatorBenchmark - Duration of single invocation 20.64 ns 75.46 ns 3.66 0.57 False
ExclusiveOrOperatorBenchmark - Duration of single invocation 20.27 ns 74.00 ns 3.65 0.61 False
BitwiseOrBenchmark - Duration of single invocation 30.62 ns 77.62 ns 2.54 0.55 False
ZeroBenchmark - Duration of single invocation 16.38 ns 22.65 ns 1.38 0.61 False
InequalityOperatorBenchmark - Duration of single invocation 61.52 ns 104.54 ns 1.70 0.48 False
BitwiseOrOperatorBenchmark - Duration of single invocation 21.13 ns 75.37 ns 3.57 0.50 False
XorBenchmark - Duration of single invocation 32.12 ns 76.60 ns 2.38 0.51 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Single>.BitwiseAndBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.47058643734464 > 33.36120757368327.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -91.76417553677699 (T) = (0 -75.65555816841108) / Math.Sqrt((3.9774959907023497 / (23)) + (0.681552268700979 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.3309856841412306 = (32.45646624221276 - 75.65555816841108) / 32.45646624221276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.CountBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.551243354083407 > 12.674870639515984.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -21.37336641130625 (T) = (0 -20.970751805771627) / Math.Sqrt((2.433710770106695 / (22)) + (0.7449289223382433 / (13))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .025) and -0.7171937024599171 = (12.212222637277653 - 20.970751805771627) / 12.212222637277653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ConditionalSelectBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 254.76672044754798 > 168.71277556763505.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -52.38464948306722 (T) = (0 -258.1582008840657) / Math.Sqrt((67.0701010948631 / (23)) + (8.828766211939513 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.6185101784627305 = (159.50360048354204 - 258.1582008840657) / 159.50360048354204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseAndOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.46425621908088 > 21.32775376924512.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -48.37642111360119 (T) = (0 -77.40782306996299) / Math.Sqrt((1.9622027385839667 / (22)) + (17.713187735174042 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -2.6672984432814575 = (21.107587579018887 - 77.40782306996299) / 21.107587579018887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ExclusiveOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 73.99774762718506 > 25.021767454195313.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -64.5118862352499 (T) = (0 -75.8278154749241) / Math.Sqrt((10.490079799255417 / (23)) + (3.2225685491314113 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -2.3874228772796937 = (22.38510461257156 - 75.8278154749241) / 22.38510461257156 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 77.61551743391695 > 32.95595925841746.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -107.89948441883425 (T) = (0 -75.96650029158153) / Math.Sqrt((2.6453102705440084 / (23)) + (0.6886902470838585 / (13))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (13) - 2, .025) and -1.3932311536828523 = (31.742232744496732 - 75.96650029158153) / 31.742232744496732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.ZeroBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 22.65184762346769 > 19.218975277997135.
IsChangePoint: Marked as a change because one of 11/2/2022 2:01:45 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -8.452645594667635 (T) = (0 -22.979729109644133) / Math.Sqrt((5.4465189379155206 / (21)) + (0.7011688861113703 / (15))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (15) - 2, .025) and -0.25550524680588 = (18.303172502151355 - 22.979729109644133) / 18.303172502151355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 104.54168386345627 > 64.05443077785903.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -26.467880396431358 (T) = (0 -105.65071909779176) / Math.Sqrt((18.757014060465192 / (23)) + (26.632983217439307 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.7036096859880056 = (62.01580090014563 - 105.65071909779176) / 62.01580090014563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.BitwiseOrOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 75.36559346823194 > 21.562070426039977.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -96.51156479783874 (T) = (0 -76.09163967912566) / Math.Sqrt((2.4195270993126345 / (22)) + (2.5911925500970003 / (12))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (12) - 2, .025) and -2.624359725148837 = (20.994505360805736 - 76.09163967912566) / 20.994505360805736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.XorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 76.59850439879277 > 33.465461952847335.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -79.21273813633783 (T) = (0 -75.67047615265449) / Math.Sqrt((3.5967639161893574 / (23)) + (1.938979301331374 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -1.3172189245392958 = (32.65573025979801 - 75.67047615265449) / 32.65573025979801 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added AOT=true PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Nov 8, 2022
@lewing
Copy link
Member

lewing commented Nov 8, 2022 •
edited
Loading

this looks very bad, commit range looks like dotnet/runtime@bdd67af...13d4e6d cc @vargaz @radekdoulik @SamMonoRT @tannergooding

@lewing
Copy link
Member

lewing commented Nov 8, 2022

what was bad for AOT was good for the interpreter #9628

@tannergooding
Copy link
Member

tannergooding commented Nov 8, 2022 •
edited
Loading

It could've potentially been dotnet/runtime#77562 but that was really just marking existing methods as [Intrinsic] so RyuJIT could avoid going through the inliner.

There were a couple software implementations that changed slightly, but that was things like EqualsAny where the software implementation doesn't rely on the compiler to treat it as intrinsic and shouldn't have impacted anything Mono or RyuJIT was already handling.

Given it impacted things like BitwiseOr which didn't change at all (just [Intrinsic] added), I'd guess it isn't that.

dotnet/runtime#77534 is the only other likely suspect that I see in that list and directly touched Mono code.

@lewing lewing mentioned this issue Nov 8, 2022
Open
@naricc naricc mentioned this issue Nov 9, 2022
Closed
This was referenced Nov 9, 2022
Open
Open
Closed
@tannergooding
Copy link
Member

As per dotnet/runtime#78163 (comment), the fixes taken here are required for correctness.

These tests should effectively be ignored on Mono until the SIMD work is completed and IsHardwareAccelerated starts reporting true. The software fallbacks are not expected to be used in production scenarios and exist purely for validation purposes and to support scenarios like the debugger.

@lewing lewing mentioned this issue Nov 10, 2022
Open
@DrewScoggins DrewScoggins mentioned this issue Sep 14, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged wasm-aot
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @tannergooding