[Perf] Changes at 7/6/2022 9:44:27 PM #6702

performanceautofiler · 2022-07-12T15:21:01Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
All_AllElementsMatch - Duration of single invocation	456.77 ns	492.44 ns	1.08	0.01	False
SelectToList - Duration of single invocation	233.59 ns	254.29 ns	1.09	0.06	False
Count - Duration of single invocation	221.30 ns	244.81 ns	1.11	0.02	False
SelectToArray - Duration of single invocation	202.96 ns	225.09 ns	1.11	0.07	False
CastToSameType - Duration of single invocation	364.01 ns	397.47 ns	1.09	0.04	False
WhereLast_LastElementMatches - Duration of single invocation	195.29 ns	213.04 ns	1.09	0.04	False
Where - Duration of single invocation	532.28 ns	576.17 ns	1.08	0.03	False
SelectToArray - Duration of single invocation	358.79 ns	380.00 ns	1.06	0.03	False
WhereSingleOrDefault_LastElementMatches - Duration of single invocation	191.89 ns	212.68 ns	1.11	0.03	False
WhereSingle_LastElementMatches - Duration of single invocation	191.11 ns	212.45 ns	1.11	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 492.43946533131816 > 479.7619987759877.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -24.283578403945892 (T) = (0 -488.8018311619306) / Math.Sqrt((0.8769003006784633 / (38)) + (65.47156722102058 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.07025734378864779 = (456.7142977329181 - 488.8018311619306) / 456.7142977329181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 254.29436248399844 > 246.60991643428125.
IsChangePoint: Marked as a change because one of 5/13/2022 5:25:17 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -14.456889640192808 (T) = (0 -253.67734753443185) / Math.Sqrt((36.65010843599415 / (38)) + (18.27756415605346 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.07355665048586359 = (236.29619118807017 - 253.67734753443185) / 236.29619118807017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 244.80996925271646 > 232.56350180502614.
IsChangePoint: Marked as a change because one of 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -36.76297102235797 (T) = (0 -243.65221270354027) / Math.Sqrt((0.11249507818584079 / (38)) + (13.990068876060276 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.10122134494032009 = (221.25634762078175 - 243.65221270354027) / 221.25634762078175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 225.08553546305728 > 213.86251752349528.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -18.985078728679365 (T) = (0 -224.20849038893547) / Math.Sqrt((29.737703278943687 / (38)) + (5.899359132891149 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.08932588373400438 = (205.82315516123688 - 224.20849038893547) / 205.82315516123688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 397.46949083338507 > 384.03849768359106.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -25.527665845367718 (T) = (0 -396.65382545632247) / Math.Sqrt((7.1079715882113295 / (38)) + (41.51144866091905 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.07851229048323197 = (367.77867897879963 - 396.65382545632247) / 367.77867897879963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 213.04423334601904 > 203.8756634543279.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -32.45783957367764 (T) = (0 -214.0825146234416) / Math.Sqrt((1.6917133873333192 / (38)) + (13.150702422605592 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.10467246179617273 = (193.79727659305294 - 214.0825146234416) / 193.79727659305294 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 576.1683263953802 > 558.7220063927178.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -33.21793979228371 (T) = (0 -574.8921819242092) / Math.Sqrt((9.26591737340947 / (38)) + (49.15056173587352 / (37))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (37) - 2, .025) and -0.07811024221560105 = (533.2406273617814 - 574.8921819242092) / 533.2406273617814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 380.002387944974 > 376.4826408634898.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -28.876930158034433 (T) = (0 -380.209731628864) / Math.Sqrt((4.966468602269173 / (38)) + (16.604134537805976 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.06069570350591366 = (358.45316462785524 - 380.209731628864) / 358.45316462785524 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 212.679234460659 > 201.30572988357582.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -34.486294074075666 (T) = (0 -211.8204110267704) / Math.Sqrt((0.7555085592070917 / (38)) + (12.511809335626682 / (37))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (37) - 2, .025) and -0.10793383291706789 = (191.18507327199364 - 211.8204110267704) / 191.18507327199364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 212.44862673884737 > 199.94807179463032.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -36.452150079307344 (T) = (0 -210.95642997531462) / Math.Sqrt((0.8466906787477068 / (38)) + (11.819186741626595 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.11081483256679255 = (189.91142699080763 - 210.95642997531462) / 189.91142699080763 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Collections.CtorFromCollectionNonGeneric<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	5.05 μs	5.73 μs	1.13	0.05	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<String>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.725103826951265 > 5.310228153271388.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -32.0420494989023 (T) = (0 -5630.528998714069) / Math.Sqrt((1269.9952587506657 / (38)) + (12746.684759981532 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.1227066703349814 = (5015.138101062578 - 5630.528998714069) / 5015.138101062578 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Tests.Perf_Char

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Char_ToLowerInvariant - Duration of single invocation	16.35 ns	17.91 ns	1.10	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.912753646038066 > 17.125645435642355.
IsChangePoint: Marked as a change because one of 5/18/2022 10:04:15 AM, 6/27/2022 8:54:23 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -25.4189199626786 (T) = (0 -17.81713546284043) / Math.Sqrt((0.026724907744736688 / (38)) + (0.12894435620837535 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.10048798931758514 = (16.19021346510912 - 17.81713546284043) / 16.19021346510912 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in Burgers

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test0 - Duration of single invocation	278.95 ms	300.98 ms	1.08	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Burgers*'

Payloads

Baseline
Compare

Histogram

Burgers.Test0

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 300.98312860000004 > 294.11983751230764.
IsChangePoint: Marked as a change because one of 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -33.08740536495802 (T) = (0 -301060296.595593) / Math.Sqrt((1358563722490.939 / (38)) + (13978892242217.078 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.075063388476464 = (280039577.03577214 - 301060296.595593) / 280039577.03577214 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
MultipleSerial - Duration of single invocation	849.19 ns	928.09 ns	1.09	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 928.0899139705881 > 913.34216374375.
IsChangePoint: Marked as a change because one of 5/15/2022 6:57:22 PM, 5/23/2022 3:43:39 PM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -11.306561092555778 (T) = (0 -924.4230147383971) / Math.Sqrt((313.5380047547589 / (38)) + (526.700240555263 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.06102306748092721 = (871.2562837424026 - 924.4230147383971) / 871.2562837424026 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
List - Duration of single invocation	569.37 ns	672.40 ns	1.18	0.00	False
IList - Duration of single invocation	1.45 μs	1.56 μs	1.08	0.05	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<String>.List(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 672.3976624962185 > 596.4247873936363.
IsChangePoint: Marked as a change because one of 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -760.3711233875302 (T) = (0 -672.3332678480899) / Math.Sqrt((0.5350248700746668 / (38)) + (0.18160372886176085 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.18386469868644614 = (567.9139420189448 - 672.3332678480899) / 567.9139420189448 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSetReverse&lt;String&gt;.IList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.564366149387796 > 1.5281118581363524.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -13.414714303919737 (T) = (0 -1704.0679899684917) / Math.Sqrt((14.412544380336056 / (38)) + (12968.08811489754 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.17028368422799384 = (1456.1153102742107 - 1704.0679899684917) / 1456.1153102742107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParseInt64 - Duration of single invocation	18.88 ns	21.96 ns	1.16	0.05	False
TryParseInt64 - Duration of single invocation	20.14 ns	22.70 ns	1.13	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.96277071608534 > 18.239750210681454.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -16.23457389045599 (T) = (0 -22.24026917654813) / Math.Sqrt((2.486929663084714 / (38)) + (1.0644449361534534 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.2872586520179707 = (17.277234176428472 - 22.24026917654813) / 17.277234176428472 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.700517140018654 > 19.54037769033696.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -13.187798467111008 (T) = (0 -22.861427434503117) / Math.Sqrt((3.244935879307557 / (38)) + (1.0853468431306088 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.24182118870029476 = (18.409596842545557 - 22.861427434503117) / 18.409596842545557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Tests.Perf_HashCode

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Add - Duration of single invocation	1.59 μs	1.68 μs	1.06	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_HashCode*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_HashCode.Add

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.679144815750798 > 1.6718925021480702.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/20/2022 10:18:01 PM, 5/25/2022 1:42:32 AM, 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -36.336157011766865 (T) = (0 -1676.067868267128) / Math.Sqrt((6.100147415588639 / (38)) + (186.20205067389077 / (37))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (37) - 2, .025) and -0.05197100044714188 = (1593.264327205515 - 1676.067868267128) / 1593.264327205515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	465fbeda31f8c55ea91f8d8c3d851a42876fa3ad
Compare	82e985c738a7207e7de99fb3202c687da9f1e264
Diff	Diff

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	3.29 μs	3.78 μs	1.15	0.07	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndRemove&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<String>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7792059695028604 > 3.555253862160572.
IsChangePoint: Marked as a change because one of 6/6/2022 10:12:27 PM, 6/20/2022 2:50:37 PM, 6/23/2022 8:35:19 AM, 7/6/2022 7:05:00 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsRegressionStdDev: Marked as regression because -15.027057670295996 (T) = (0 -3648.8162636211973) / Math.Sqrt((3274.1565039302945 / (38)) + (11244.251781079642 / (38))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (38) - 2, .025) and -0.08754621705915856 = (3355.0907597177684 - 3648.8162636211973) / 3355.0907597177684 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

AndyAyersMS · 2022-07-21T23:04:24Z

Likely from dotnet/runtime#71659

performanceautofiler bot assigned tannergooding Jul 12, 2022

performanceautofiler bot added CoreClr PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Jul 12, 2022

DrewScoggins mentioned this issue Jul 12, 2022

[Perf] Regressions in System.Tests.Perf_String dotnet/runtime#72026

Closed

AndyAyersMS removed the untriaged label Jul 21, 2022

AndyAyersMS closed this as completed Jul 21, 2022

performanceautofiler bot added perf-regression arch-x64 os-linux branch-refs/heads/main runtime-coreclr kind-micro and removed refs/heads/main labels Mar 24, 2023

[Perf] Changes at 7/6/2022 9:44:27 PM #6702

[Perf] Changes at 7/6/2022 9:44:27 PM #6702

Comments

performanceautofiler bot commented Jul 12, 2022 • edited Loading

Run Information

Regressions in System.Linq.Tests.Perf_Enumerable

Repro

Payloads

Histogram

System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Collections.CtorFromCollectionNonGeneric<String>

Repro

Payloads

Histogram

System.Collections.CtorFromCollectionNonGeneric<String>.Queue(Size: 512)

Description of detection logic

Docs

Run Information

Regressions in System.Tests.Perf_Char

Repro

Payloads

Histogram

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")

Description of detection logic

Docs

Run Information

Regressions in Burgers

Repro

Payloads

Histogram

Burgers.Test0

Description of detection logic

Docs

Run Information

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Repro

Payloads

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

Description of detection logic

Docs

Run Information

Regressions in System.Collections.IndexerSetReverse<String>

Repro

Payloads

Histogram

System.Collections.IndexerSetReverse<String>.List(Size: 512)

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Repro

Payloads

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Tests.Perf_HashCode

Repro

Payloads

Histogram

System.Tests.Perf_HashCode.Add

Description of detection logic

Docs

Run Information

Regressions in System.Collections.CreateAddAndRemove<String>

performanceautofiler bot commented Jul 12, 2022 •

edited

Loading