[Perf] Linux/x64: 70 Regressions on 4/17/2023 11:29:15 PM #16707

performanceautofiler · 2023-04-19T15:20:13Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	47.15 ns	66.65 ns	1.41	0.49	False
ParseSpan - Duration of single invocation	46.28 ns	55.92 ns	1.21	0.61	False
ToString - Duration of single invocation	93.81 ns	177.26 ns	1.89	0.21	False
TryFormat - Duration of single invocation	58.62 ns	142.93 ns	2.44	0.35	False
Parse - Duration of single invocation	54.50 ns	75.82 ns	1.39	0.52	False
ParseSpan - Duration of single invocation	56.86 ns	70.75 ns	1.24	0.49	False
Parse - Duration of single invocation	32.90 ns	58.54 ns	1.78	0.66	False
ParseSpan - Duration of single invocation	54.71 ns	73.80 ns	1.35	0.47	False
TryFormat - Duration of single invocation	43.11 ns	132.40 ns	3.07	0.43	False
TryParseSpan - Duration of single invocation	50.60 ns	65.41 ns	1.29	0.49	False
TryParseSpan - Duration of single invocation	42.89 ns	72.66 ns	1.69	0.46	False
Parse - Duration of single invocation	53.13 ns	70.74 ns	1.33	0.54	False
TryParse - Duration of single invocation	33.57 ns	42.86 ns	1.28	0.61	False
TryParse - Duration of single invocation	44.40 ns	59.61 ns	1.34	0.50	False
ToString - Duration of single invocation	87.57 ns	167.90 ns	1.92	0.26	False
TryParseSpan - Duration of single invocation	32.73 ns	49.49 ns	1.51	0.63	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.65407948353578 > 49.08573500759305.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.214734406323865 (T) = (0 -64.75726607888892) / Math.Sqrt((6.427553433777207 / (31)) + (29.856855465460796 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.3620502819201144 = (47.543961437017636 - 64.75726607888892) / 47.543961437017636 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.ParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.91660181778166 > 44.39230450637475.
IsChangePoint: Marked as a change because one of 3/1/2023 8:53:57 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -11.130609501457226 (T) = (0 -53.489668485057535) / Math.Sqrt((7.6399459657311315 / (31)) + (5.470068751572079 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.3593612656300535 = (39.34911920582457 - 53.489668485057535) / 39.34911920582457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 177.25626320668863 > 101.95593389960267.
IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/12/2023 8:39:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -43.91086978136453 (T) = (0 -183.25654299324643) / Math.Sqrt((20.884496472543468 / (27)) + (22.612033857954785 / (7))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (7) - 2, .025) and -0.9210649920849082 = (95.39320311821432 - 183.25654299324643) / 95.39320311821432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.93373920478072 > 63.61616699551488.
IsChangePoint: Marked as a change because one of 2/12/2023 9:12:37 PM, 4/12/2023 8:39:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -40.740214778109994 (T) = (0 -150.23926998075237) / Math.Sqrt((6.112168260463308 / (27)) + (37.117066530547916 / (8))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (8) - 2, .025) and -1.488629229363863 = (60.37029068374164 - 150.23926998075237) / 60.37029068374164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.Parse(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.81656617966226 > 55.42706771231794.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -11.634474169791803 (T) = (0 -74.4314939681714) / Math.Sqrt((14.34480560637661 / (31)) + (11.685152169244361 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.40359859652289226 = (53.029045592207844 - 74.4314939681714) / 53.029045592207844 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.7513529127829 > 55.826118148138725.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -8.00987340719528 (T) = (0 -66.5724695286111) / Math.Sqrt((4.510019536821651 / (31)) + (8.047777502804928 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.21466170109235178 = (54.8074163108478 - 66.5724695286111) / 54.8074163108478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.54356172914721 > 39.30962340642522.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.836582930922451 (T) = (0 -60.84140075663737) / Math.Sqrt((8.828700784126283 / (31)) + (49.65102239088642 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.6677934442931922 = (36.48017742534169 - 60.84140075663737) / 36.48017742534169 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.79555100833508 > 57.34675419485304.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.133211316799 (T) = (0 -72.1111607212261) / Math.Sqrt((6.656040357331825 / (30)) + (22.279392727107417 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.31243912093259574 = (54.94438528317047 - 72.1111607212261) / 54.94438528317047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.39613571490602 > 44.09204420183327.
IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/12/2023 8:39:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -59.09804589611238 (T) = (0 -128.19799512371432) / Math.Sqrt((5.189794734735334 / (26)) + (15.02713127928375 / (8))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (8) - 2, .025) and -1.9808918326876495 = (43.00659075177769 - 128.19799512371432) / 43.00659075177769 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 65.40644534823915 > 50.616588869231386.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.704273792985693 (T) = (0 -65.9797176368667) / Math.Sqrt((9.983020159424665 / (29)) + (19.100955662302628 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.3590665473102058 = (48.54781965419599 - 65.9797176368667) / 48.54781965419599 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 72.65732829187229 > 47.24801122034719.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.083475288286967 (T) = (0 -65.60235995114638) / Math.Sqrt((3.657763133212528 / (30)) + (28.829094137846567 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.4130635460997811 = (46.42562617386634 - 65.60235995114638) / 46.42562617386634 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.74270614035089 > 55.04564984069868.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.131900396736497 (T) = (0 -79.21583509550265) / Math.Sqrt((9.62738834547704 / (31)) + (56.83139013305247 / (3))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .025) and -0.5143771215304647 = (52.30918637719863 - 79.21583509550265) / 52.30918637719863 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.86193450160554 > 32.614808339873.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -8.36579427345511 (T) = (0 -44.469023743143275) / Math.Sqrt((5.304595746027721 / (30)) + (8.649872155109973 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.4039701773079578 = (31.673766624025014 - 44.469023743143275) / 31.673766624025014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.60781691761335 > 48.62719111068348.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -8.23842249779232 (T) = (0 -62.60958792066222) / Math.Sqrt((7.666468039643136 / (31)) + (12.79344541567219 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.3231940846291856 = (47.31701014081245 - 62.60958792066222) / 47.31701014081245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.90080133645174 > 86.99845524232859.
IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/12/2023 8:39:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -61.71347684158116 (T) = (0 -163.7814615163517) / Math.Sqrt((13.95897289532938 / (27)) + (9.962192747995944 / (8))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (8) - 2, .025) and -1.0008380813240985 = (81.85642958572926 - 163.7814615163517) / 81.85642958572926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.48755981327589 > 33.411622657171975.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -8.929654702660612 (T) = (0 -51.097226870042995) / Math.Sqrt((3.4417124882538324 / (29)) + (19.625150314284408 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.6440485187646825 = (31.0801209859894 - 51.097226870042995) / 31.0801209859894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	24.72 ns	33.23 ns	1.34	0.72	False
Parse - Duration of single invocation	39.86 ns	58.45 ns	1.47	0.52	False
TryParse - Duration of single invocation	27.09 ns	44.17 ns	1.63	0.63	False
Parse - Duration of single invocation	31.51 ns	48.33 ns	1.53	0.61	False
Parse - Duration of single invocation	30.68 ns	51.83 ns	1.69	0.66	False
ParseSpan - Duration of single invocation	32.39 ns	38.92 ns	1.20	0.57	False
TryParse - Duration of single invocation	35.70 ns	48.88 ns	1.37	0.65	False
ParseSpan - Duration of single invocation	43.67 ns	56.05 ns	1.28	0.50	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.227087047278886 > 25.790554941519755.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -3.4715399348360707 (T) = (0 -35.43222089583031) / Math.Sqrt((6.749545048645538 / (31)) + (27.46055719976979 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.3527267522158456 = (26.193184128125107 - 35.43222089583031) / 26.193184128125107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt32.Parse(value: "4294967295")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.446618575455666 > 41.63456134248349.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.843104545558493 (T) = (0 -60.79785416612758) / Math.Sqrt((9.307047926788732 / (31)) + (23.30495352683089 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.46908521560592964 = (41.38483834721002 - 60.79785416612758) / 41.38483834721002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt32.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.16670206983306 > 30.75520699345637.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.1624987494019035 (T) = (0 -39.874739713560686) / Math.Sqrt((5.110845290405155 / (31)) + (11.095275023145035 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.3604195563294709 = (29.310619307139458 - 39.874739713560686) / 29.310619307139458 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.32545693068112 > 36.87994111871435.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -9.887453654157152 (T) = (0 -50.39125033200156) / Math.Sqrt((6.864754652675831 / (31)) + (7.365733112549131 / (3))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .025) and -0.47279120655130513 = (34.21479576185001 - 50.39125033200156) / 34.21479576185001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt32.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.82907352838014 > 32.21919879547389.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -13.042671592178108 (T) = (0 -48.794819339724114) / Math.Sqrt((5.231194112515396 / (31)) + (6.3595210876280435 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.5491154823078166 = (31.49850343438008 - 48.794819339724114) / 31.49850343438008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt32.ParseSpan(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.916611129921414 > 34.059672441367425.
IsChangePoint: Marked as a change because one of 2/14/2023 9:29:25 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -3.2952241417165866 (T) = (0 -40.519951051753814) / Math.Sqrt((4.2091470998878915 / (31)) + (15.379926079704488 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.1936811775322878 = (33.945371523341954 - 40.519951051753814) / 33.945371523341954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt32.TryParse(value: "4294967295")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.877732509081554 > 39.45497317178227.
IsChangePoint: Marked as a change because one of 1/30/2023 9:42:46 PM, 1/31/2023 8:52:14 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.8546022282062715 (T) = (0 -47.03057207626678) / Math.Sqrt((13.882335218674397 / (31)) + (3.6315543516170035 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.2043855953724093 = (39.04943089403557 - 47.03057207626678) / 39.04943089403557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt32.ParseSpan(value: "4294967295")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.050502178189994 > 45.73901408020885.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -3.8572432549173876 (T) = (0 -52.73021570329765) / Math.Sqrt((12.93017369324685 / (30)) + (16.027297317550254 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.18217292209816568 = (44.60448612687732 - 52.73021570329765) / 44.60448612687732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-04-19T15:20:17Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Get - Duration of single invocation	5.25 ms	5.89 ms	1.12	0.14	False
Get - Duration of single invocation	15.17 ms	17.51 ms	1.15	0.14	False
Get - Duration of single invocation	8.08 ms	9.10 ms	1.13	0.13	False
Get - Duration of single invocation	17.53 ms	19.76 ms	1.13	0.13	False
Get - Duration of single invocation	69.81 ms	83.25 ms	1.19	0.17	False
Get - Duration of single invocation	3.74 ms	4.09 ms	1.09	0.10	False
Get - Duration of single invocation	1.17 ms	1.26 ms	1.08	0.11	False
Get - Duration of single invocation	32.79 ms	37.29 ms	1.14	0.11	False
Get - Duration of single invocation	2.49 ms	2.70 ms	1.09	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 40)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.886675168367347 > 5.508982659361702.
IsChangePoint: Marked as a change because one of 3/10/2023 8:23:08 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -8.663282054320563 (T) = (0 -5831595.36533167) / Math.Sqrt((19154656779.595745 / (30)) + (13156238605.665176 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.10265900819922141 = (5288666.14426466 - 5831595.36533167) / 5288666.14426466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.5061381 > 15.854764992499998.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -13.580950359985936 (T) = (0 -17172900.00357143) / Math.Sqrt((122493021916.38423 / (30)) + (85405255733.21288 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.14432698294023905 = (15006986.866155423 - 17172900.00357143) / 15006986.866155423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 20)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.09955556084656 > 8.4952177415.
IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.605836899265413 (T) = (0 -9150665.196587255) / Math.Sqrt((34451336923.237335 / (31)) + (78746617482.45705 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.13619382841033464 = (8053788.858710916 - 9150665.196587255) / 8053788.858710916 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 40)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.75903636969697 > 18.5634954225.
IsChangePoint: Marked as a change because one of 3/9/2023 8:36:25 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -4.846491714301326 (T) = (0 -20160617.66919747) / Math.Sqrt((263776696311.76303 / (31)) + (886223553395.4121 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.1303332569593183 = (17835994.424715992 - 20160617.66919747) / 17835994.424715992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 40)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.24710003333333 > 75.20264063437499.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -12.394106941820988 (T) = (0 -81822308.93191391) / Math.Sqrt((3798965183081.7197 / (29)) + (2194778611195.9714 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.14270311861116577 = (71604170.49649802 - 81822308.93191391) / 71604170.49649802 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.093113491803279 > 3.8621493336779174.
IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -9.23199710797045 (T) = (0 -4077070.106684231) / Math.Sqrt((7151899007.961786 / (30)) + (6274176115.957385 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.10650576251346545 = (3684635.2227059603 - 4077070.106684231) / 3684635.2227059603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2626811480117819 > 1.2358503904493343.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -4.6779120904195155 (T) = (0 -1316364.2070178266) / Math.Sqrt((507669303.5017984 / (31)) + (3189657610.928011 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.11281167885099726 = (1182917.3183885003 - 1316364.2070178266) / 1182917.3183885003 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 20)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.28628571428572 > 34.08206858642857.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -10.355085315883633 (T) = (0 -37317763.36805555) / Math.Sqrt((402371430104.6088 / (29)) + (824930904812.1688 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.14966763756036988 = (32459610.19416446 - 37317763.36805555) / 32459610.19416446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 20)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.704889629598662 > 2.598581689554265.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -9.119231724385322 (T) = (0 -2752341.6522199186) / Math.Sqrt((5455626461.502614 / (31)) + (2239071126.787864 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.09874613262154837 = (2504984.1546681775 - 2752341.6522199186) / 2504984.1546681775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ParseHex - Duration of single invocation	31.97 ns	50.25 ns	1.57	0.59	False
ParseSpan - Duration of single invocation	32.48 ns	38.48 ns	1.18	0.63	False
ParseHex - Duration of single invocation	29.81 ns	57.11 ns	1.92	0.60	False
TryParseSpan - Duration of single invocation	26.95 ns	39.10 ns	1.45	0.65	False
ParseHex - Duration of single invocation	37.62 ns	51.52 ns	1.37	0.64	False
TryParse - Duration of single invocation	34.96 ns	46.03 ns	1.32	0.60	False
TryParse - Duration of single invocation	27.34 ns	39.37 ns	1.44	0.66	False
Parse - Duration of single invocation	39.71 ns	56.64 ns	1.43	0.58	False
ParseSpan - Duration of single invocation	41.76 ns	49.82 ns	1.19	0.52	False
TryParseSpan - Duration of single invocation	33.69 ns	42.20 ns	1.25	0.62	False
Parse - Duration of single invocation	32.22 ns	52.97 ns	1.64	0.65	False
TryParse - Duration of single invocation	38.68 ns	44.76 ns	1.16	0.64	False
TryParseSpan - Duration of single invocation	27.22 ns	35.84 ns	1.32	0.68	False
Parse - Duration of single invocation	34.30 ns	46.80 ns	1.36	0.60	False
ParseHex - Duration of single invocation	39.33 ns	59.18 ns	1.50	0.59	False
TryParse - Duration of single invocation	28.36 ns	32.73 ns	1.15	0.75	False
Parse - Duration of single invocation	39.84 ns	53.02 ns	1.33	0.62	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ParseHex(value: "3039")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.25033412468357 > 33.30635004067855.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -9.689514963785298 (T) = (0 -54.187440694602095) / Math.Sqrt((5.022197379943919 / (31)) + (17.439394791467574 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.6135367436579894 = (33.58302245522823 - 54.187440694602095) / 33.58302245522823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.ParseSpan(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.481924259829746 > 35.20735146974485.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -5.079078124412205 (T) = (0 -40.90627721623721) / Math.Sqrt((7.534339085899918 / (31)) + (4.762835290294006 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.17463670310357285 = (34.824620334233096 - 40.90627721623721) / 34.824620334233096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.ParseHex(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.10575078895138 > 31.247321350923475.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.041631509393214 (T) = (0 -50.55844918905853) / Math.Sqrt((8.355093450487756 / (30)) + (34.8090774244047 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.5579011045662445 = (32.4529259533038 - 50.55844918905853) / 32.4529259533038 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.TryParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.10087703499541 > 28.294668909858213.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -12.755134058567608 (T) = (0 -38.38665524879708) / Math.Sqrt((5.625116238000599 / (31)) + (1.4972938820350221 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.32928692795306497 = (28.8776293827757 - 38.38665524879708) / 28.8776293827757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.52244593809881 > 41.19296492157341.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -4.613651065262631 (T) = (0 -57.12744351248118) / Math.Sqrt((12.397368444116184 / (31)) + (47.21468223832397 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.39300283494938193 = (41.01028517616552 - 57.12744351248118) / 41.01028517616552 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.TryParse(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.03033142413826 > 39.11795579853169.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -17.431049383527206 (T) = (0 -45.2116440753695) / Math.Sqrt((3.499486347832268 / (30)) + (0.3520736800203823 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.2112707646174161 = (37.3257948561565 - 45.2116440753695) / 37.3257948561565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.36638358020323 > 29.174051044530582.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.8690404051664995 (T) = (0 -41.71897066282998) / Math.Sqrt((3.556421773394703 / (29)) + (13.259188915860237 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.43940148443951027 = (28.9835540075707 - 41.71897066282998) / 28.9835540075707 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.Parse(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.637572528454335 > 41.85268814717923.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -5.7544602474034585 (T) = (0 -56.917132176058125) / Math.Sqrt((7.582435186039759 / (30)) + (30.774782493004228 / (5))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (5) - 2, .025) and -0.3439506542803314 = (42.35061160525758 - 56.917132176058125) / 42.35061160525758 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.ParseSpan(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.81564595985591 > 43.77949777728545.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -8.846704668307591 (T) = (0 -51.99903873136902) / Math.Sqrt((3.9139688388148053 / (30)) + (3.784938470920071 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.21438237366947313 = (42.81932928114284 - 51.99903873136902) / 42.81932928114284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.20199783639912 > 36.02986514954454.
IsChangePoint: Marked as a change because one of 2/12/2023 9:12:37 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -5.009559241376697 (T) = (0 -45.25722781567036) / Math.Sqrt((7.966358377085001 / (31)) + (13.820483112345519 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.27107747325374193 = (35.60540468066007 - 45.25722781567036) / 35.60540468066007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.Parse(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.96887929193893 > 33.82207317260812.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -15.15722731514657 (T) = (0 -50.442745082518584) / Math.Sqrt((5.99816019471863 / (30)) + (5.366388941628088 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.5950909046343861 = (31.62374315843815 - 50.442745082518584) / 31.62374315843815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.TryParse(value: "-2147483648")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.761273066569856 > 40.80844944456417.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -4.489029511013598 (T) = (0 -46.33542076914553) / Math.Sqrt((7.071116677181677 / (30)) + (10.000618148001623 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.19082438876665356 = (38.910372684872115 - 46.33542076914553) / 38.910372684872115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.TryParseSpan(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.839003379113514 > 29.25305139813493.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -11.178785641328902 (T) = (0 -35.84560319471626) / Math.Sqrt((6.737157203693724 / (31)) + (1.7285427529371509 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.33569343468306445 = (26.83669939818321 - 35.84560319471626) / 26.83669939818321 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.80335916510853 > 35.626828870649625.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -4.861613692911571 (T) = (0 -54.63962435418782) / Math.Sqrt((6.640609771949016 / (31)) + (64.20259603013568 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.5596672397269833 = (35.03287301447223 - 54.63962435418782) / 35.03287301447223 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.ParseHex(value: "80000000")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.175597178855305 > 41.355521956103026.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.439238825929638 (T) = (0 -59.60393023779904) / Math.Sqrt((18.362505085770508 / (30)) + (22.09324844238021 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.44750062955063635 = (41.17713596870942 - 59.60393023779904) / 41.17713596870942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.TryParse(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.73122850490463 > 27.46044267264574.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.305638040994319 (T) = (0 -33.808080087853305) / Math.Sqrt((5.017914933563412 / (30)) + (3.2264034048674173 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.2710501381737709 = (26.598541688078754 - 33.808080087853305) / 26.598541688078754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int32.Parse(value: "2147483647")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.0201879236812 > 43.00642028122174.
IsChangePoint: Marked as a change because one of 2/14/2023 9:29:25 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -7.12401214630086 (T) = (0 -58.912241006732565) / Math.Sqrt((10.524161561288178 / (31)) + (21.425260087909948 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.40567934092870905 = (41.91015638588686 - 58.912241006732565) / 41.91015638588686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-04-19T15:20:18Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	35.16 ns	41.04 ns	1.17	0.61	False
Parse - Duration of single invocation	32.63 ns	50.39 ns	1.54	0.64	False
Parse - Duration of single invocation	34.50 ns	48.68 ns	1.41	0.60	False
TryParse - Duration of single invocation	32.65 ns	46.91 ns	1.44	0.62	False
Parse - Duration of single invocation	32.72 ns	63.53 ns	1.94	0.64	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.TryParse(value: "-32768")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.036167460423954 > 37.54410861650342.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -3.8982117967939827 (T) = (0 -44.48445104654185) / Math.Sqrt((6.394977717684011 / (31)) + (17.314881084192816 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.22942994518716245 = (36.18298970241021 - 44.48445104654185) / 36.18298970241021 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.385920738102975 > 31.68808467436863.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -13.905355522488112 (T) = (0 -47.46086685757125) / Math.Sqrt((5.2966039695329945 / (30)) + (4.804141339090456 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.5241101821842266 = (31.140049723671765 - 47.46086685757125) / 31.140049723671765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int16.Parse(value: "32767")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.681587176245635 > 35.15526808425433.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.350454105005417 (T) = (0 -55.213437655639865) / Math.Sqrt((4.229719910248738 / (30)) + (41.258457575009594 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.5921214035101291 = (34.67916299216349 - 55.213437655639865) / 34.67916299216349 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int16.TryParse(value: "32767")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.9094796060868 > 34.677916872119944.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -6.188543393311778 (T) = (0 -42.96294230904961) / Math.Sqrt((5.19545448312701 / (29)) + (7.979228747507742 / (5))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (5) - 2, .025) and -0.23748316151236942 = (34.71800154156697 - 42.96294230904961) / 34.71800154156697 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Int16.Parse(value: "-32768")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.525243659994835 > 36.65018292827902.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -9.985469055592118 (T) = (0 -60.229851041031566) / Math.Sqrt((10.203768560488497 / (31)) + (21.398265564996034 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.6531033789234998 = (36.43441288000586 - 60.229851041031566) / 36.43441288000586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse - Duration of single invocation	33.21 ns	53.58 ns	1.61	0.65	False
TryParse - Duration of single invocation	34.70 ns	44.54 ns	1.28	0.63	False
Parse - Duration of single invocation	32.97 ns	58.99 ns	1.79	0.66	False
Parse - Duration of single invocation	37.43 ns	53.90 ns	1.44	0.65	False
TryParse - Duration of single invocation	32.56 ns	37.13 ns	1.14	0.66	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.58435047836433 > 34.94967253936497.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -9.255921888878461 (T) = (0 -55.30101495128676) / Math.Sqrt((7.811878315049696 / (29)) + (18.52412527549072 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.5885933901703322 = (34.81130872976708 - 55.30101495128676) / 34.81130872976708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt16.TryParse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.54190090042888 > 38.016056260212565.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -5.364544856946386 (T) = (0 -44.446091738281915) / Math.Sqrt((7.220774140846915 / (31)) + (10.263016945055599 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.2530048640782415 = (35.47160351287077 - 44.446091738281915) / 35.47160351287077 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.99058563365382 > 32.796857194439674.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -5.142220348718134 (T) = (0 -49.30743728268132) / Math.Sqrt((4.77089482736475 / (30)) + (47.40710208284167 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.5659992703772316 = (31.48624537405034 - 49.30743728268132) / 31.48624537405034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt16.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.8994943357482 > 36.8968135067915.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -11.710915060370974 (T) = (0 -51.62347656966884) / Math.Sqrt((12.196376802431823 / (29)) + (6.522226734524101 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.48124488907499124 = (34.85141245071684 - 51.62347656966884) / 34.85141245071684 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_UInt16.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.13168804192339 > 34.07391004759547.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -3.4868065632948237 (T) = (0 -41.63717874584094) / Math.Sqrt((3.8871819846503297 / (29)) + (16.268841337015846 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.20721460369737504 = (34.4902874918158 - 41.63717874584094) / 34.4902874918158 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Benchmark.GetChildKeysTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
AddChainedConfigurationNoDelimiter - Duration of single invocation	1.71 ms	2.05 ms	1.20	0.17	False
AddChainedConfigurationWithSplitting - Duration of single invocation	922.84 μs	1.07 ms	1.16	0.14	False
AddChainedConfigurationWithCommonPaths - Duration of single invocation	967.38 μs	1.09 ms	1.12	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0512980775240384 > 1.784759933934295.
IsChangePoint: Marked as a change because one of 3/1/2023 8:53:57 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -18.438189965174264 (T) = (0 -2036925.2870557178) / Math.Sqrt((2155959509.8134456 / (30)) + (908235710.1201237 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.185541223144948 = (1718139.5697504792 - 2036925.2870557178) / 1718139.5697504792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Benchmark.GetChildKeysTests.AddChainedConfigurationWithSplitting

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0716105561111111 > 978.153858957862.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -14.468938767711233 (T) = (0 -1086968.6018869558) / Math.Sqrt((830053610.5718349 / (31)) + (219139301.98309898 / (3))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .025) and -0.15339518332683633 = (942407.7866804674 - 1086968.6018869558) / 942407.7866804674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Benchmark.GetChildKeysTests.AddChainedConfigurationWithCommonPaths

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0875163714285714 > 974.1990836085972.
IsChangePoint: Marked as a change because one of 3/6/2023 4:48:38 PM, 3/7/2023 4:52:41 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -20.80108412446269 (T) = (0 -1077629.0052086876) / Math.Sqrt((419991338.4103801 / (31)) + (133608069.33119337 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.15242157906320078 = (935099.6413002681 - 1077629.0052086876) / 935099.6413002681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	30.80 ns	41.62 ns	1.35	0.69	False
TryParse - Duration of single invocation	33.08 ns	51.32 ns	1.55	0.62	False
Parse - Duration of single invocation	31.78 ns	56.43 ns	1.78	0.60	False
Parse - Duration of single invocation	32.59 ns	58.97 ns	1.81	0.66	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.TryParse(value: "127")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.624405946019124 > 33.77374419893285.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -3.804987391531826 (T) = (0 -41.60461865305588) / Math.Sqrt((6.692474002189926 / (30)) + (16.029563994384603 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.23169019051445147 = (33.77847690390267 - 41.60461865305588) / 33.77847690390267 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_SByte.TryParse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.315893531657316 > 33.26698655792125.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -4.261273021395898 (T) = (0 -42.984554368707464) / Math.Sqrt((6.818554490414939 / (30)) + (22.367852209200663 / (5))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (5) - 2, .025) and -0.2737864989746214 = (33.74549377255087 - 42.984554368707464) / 33.74549377255087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_SByte.Parse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.43166397237814 > 35.64176366722316.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -12.397154169993232 (T) = (0 -53.828017943638926) / Math.Sqrt((5.476405378831438 / (30)) + (10.056447793627157 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.6082456203556604 = (33.47002302529819 - 53.828017943638926) / 33.47002302529819 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_SByte.Parse(value: "127")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.97263992240596 > 33.35013330334558.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -10.855367076781619 (T) = (0 -55.03990326629497) / Math.Sqrt((3.433523236538163 / (31)) + (17.781707772390796 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.727073694832194 = (31.868879383078532 - 55.03990326629497) / 31.868879383078532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse4 - Duration of single invocation	209.15 ns	246.72 ns	1.18	0.26	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.Parse4

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 246.72414804685982 > 220.43254047769202.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 2/25/2023 3:43:07 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -4.664904832742681 (T) = (0 -247.76669575621662) / Math.Sqrt((79.98263106990969 / (31)) + (199.49838175963123 / (4))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (4) - 2, .025) and -0.15789196164759342 = (213.98084101358057 - 247.76669575621662) / 213.98084101358057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Compare	eb71e484d2d4a7ae002ea524d2edead773289bbb
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Byte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	32.53 ns	47.92 ns	1.47	0.66	False
Parse - Duration of single invocation	35.03 ns	44.06 ns	1.26	0.69	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.Parse(value: "255")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.92379694598687 > 33.98660672193688.
IsChangePoint: Marked as a change because one of 2/14/2023 9:29:25 PM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -22.101220728316232 (T) = (0 -48.83534940211309) / Math.Sqrt((8.66282974978097 / (30)) + (0.9212173742163055 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.4838040968835282 = (32.91226214072547 - 48.83534940211309) / 32.91226214072547 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Tests.Perf_Byte.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.05953394278741 > 32.13360356781855.
IsChangePoint: Marked as a change because one of 2/16/2023 3:28:45 AM, 4/17/2023 1:54:35 PM, 4/19/2023 1:39:34 AM falls between 4/10/2023 1:33:42 PM and 4/19/2023 1:39:34 AM.
IsRegressionStdDev: Marked as regression because -13.784047696371932 (T) = (0 -48.21111656387499) / Math.Sqrt((6.988764064363124 / (30)) + (6.2005946925243025 / (5))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (5) - 2, .025) and -0.5314178083674788 = (31.481360802032842 - 48.21111656387499) / 31.481360802032842 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

lewing · 2023-04-20T05:14:45Z

@kg dotnet/runtime#84587

kg · 2023-04-20T19:13:38Z

This is AOT, I think? So I probably can't fix it.

lewing · 2023-04-20T19:36:35Z

this looks like dotnet/runtime#84582

cc @vargaz @tannergooding

lewing · 2023-04-20T19:37:59Z

This is AOT, I think? So I probably can't fix it.

I think I commented on the wrong pr last night

tannergooding · 2023-04-20T19:47:31Z

The TryParse ones are, the others are likely the format changes @stephentoub's been working on.

Numbers look a bit worse than we saw locally, particularly for Int64 and ParseHex. For Int64 similar question as #16604 on if there is maybe some missing handling for Int64 that exists for the smaller types. We'd expect a similarly scaled difference but the 64-bit ones are significant outliers.

ParseHex might be just a timing conflict between the different PRs Stephen and I have done and may be resolved with dotnet/runtime#84998 as it uses the TChar.CastFrom helper to avoid the more costly CreateTruncating.

performanceautofiler bot added arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression untriaged wasm-aot labels Apr 19, 2023

radekdoulik assigned kg Apr 20, 2023

radekdoulik removed the untriaged label Apr 20, 2023

[Perf] Linux/x64: 70 Regressions on 4/17/2023 11:29:15 PM #16707

[Perf] Linux/x64: 70 Regressions on 4/17/2023 11:29:15 PM #16707

Comments

performanceautofiler bot commented Apr 19, 2023 • edited Loading

Run Information

Regressions in System.Tests.Perf_Int64

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")

Description of detection logic

System.Tests.Perf_Int64.ParseSpan(value: "12345")

Description of detection logic

System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

Description of detection logic

System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

Description of detection logic

System.Tests.Perf_Int64.Parse(value: "9223372036854775807")

Description of detection logic

System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

Description of detection logic

System.Tests.Perf_Int64.Parse(value: "12345")

Description of detection logic

System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")

Description of detection logic

System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

Description of detection logic

System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

Description of detection logic

System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")

Description of detection logic

System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")

Description of detection logic

System.Tests.Perf_Int64.TryParse(value: "12345")

Description of detection logic

System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

Description of detection logic

System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

Description of detection logic

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

Description of detection logic

Docs

Run Information

Regressions in System.Tests.Perf_UInt32

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_UInt32.TryParse(value: "0")

Description of detection logic

System.Tests.Perf_UInt32.Parse(value: "4294967295")

Description of detection logic

System.Tests.Perf_UInt32.TryParse(value: "12345")

Description of detection logic

System.Tests.Perf_UInt32.Parse(value: "12345")

Description of detection logic

System.Tests.Perf_UInt32.Parse(value: "0")

Description of detection logic

System.Tests.Perf_UInt32.ParseSpan(value: "0")

Description of detection logic

System.Tests.Perf_UInt32.TryParse(value: "4294967295")

Description of detection logic

System.Tests.Perf_UInt32.ParseSpan(value: "4294967295")

Description of detection logic

Docs

performanceautofiler bot commented Apr 19, 2023 • edited Loading

Run Information

Regressions in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks

Repro

Payloads

Payloads

Histogram

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 40)

Description of detection logic

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 10)

Description of detection logic

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 20)

Description of detection logic

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 40)

performanceautofiler bot commented Apr 19, 2023 •

edited

Loading

performanceautofiler bot commented Apr 19, 2023 •

edited

Loading

performanceautofiler bot commented Apr 19, 2023 •

edited

Loading