[Perf] Changes at 7/15/2022 7:20:55 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	68e13b68bc20a36a0cfd715d88cf04e4fb9b510b
Compare	00d9488a70ad6e4e8d7aa6e0325d0deba0a14957
Diff	Diff

Improvements in System.IO.Compression.Brotli

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Decompress_WithState - Duration of single invocation	698.74 μs	287.54 μs	0.41	0.00	False
Decompress - Duration of single invocation	705.63 μs	295.68 μs	0.42	0.00	False
Decompress_WithoutState - Duration of single invocation	380.16 μs	331.48 μs	0.87	0.00	False
Decompress_WithoutState - Duration of single invocation	131.37 μs	106.33 μs	0.81	0.00	False
Decompress - Duration of single invocation	398.78 μs	348.07 μs	0.87	0.00	False
Decompress_WithState - Duration of single invocation	380.17 μs	332.84 μs	0.88	0.00	False
Decompress_WithState - Duration of single invocation	131.99 μs	106.99 μs	0.81	0.00	False
Decompress_WithoutState - Duration of single invocation	695.33 μs	289.06 μs	0.42	0.00	False
Decompress - Duration of single invocation	136.63 μs	111.45 μs	0.82	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Compression.Brotli*'

Payloads

Baseline
Compare

Histogram

System.IO.Compression.Brotli.Decompress_WithState(level: Optimal, file: "TestDocument.pdf")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 287.53658333333334 < 662.715372006689.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 1748.0926593612644 (T) = (0 -288620.0795706074) / Math.Sqrt((3448781.6101011615 / (299)) + (604827.2708847492 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.586267265798033 = (697600.3001728047 - 288620.0795706074) / 697600.3001728047 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "TestDocument.pdf")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 295.6778259214744 < 676.1840551318472.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 1422.0393682716235 (T) = (0 -295587.5182967262) / Math.Sqrt((20819317.088080857 / (299)) + (251176.22028412033 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.5874005128016636 = (716403.0190726762 - 295587.5182967262) / 716403.0190726762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "alice29.txt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 331.4836833333334 < 361.61958556928715.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 266.0165791333072 (T) = (0 -332064.7881251884) / Math.Sqrt((3058347.548360813 / (299)) + (334013.99222890555 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.12884665638594747 = (381178.34312337 - 332064.7881251884) / 381178.34312337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "sum")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.33101496598641 < 124.66981317649926.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 189.40995818784646 (T) = (0 -107158.54869130815) / Math.Sqrt((301921.04491196654 / (299)) + (214522.89658022378 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.18426949822446903 = (131365.13647346187 - 107158.54869130815) / 131365.13647346187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "alice29.txt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 348.0687327991453 < 378.05747601376476.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 167.95342097854544 (T) = (0 -346639.63964033674) / Math.Sqrt((1414867.5883926004 / (299)) + (1242695.5346873929 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1290350625652335 = (397994.9418644644 - 346639.63964033674) / 397994.9418644644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress_WithState(level: Optimal, file: "alice29.txt")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 332.83989453764326 < 361.77992837053563.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 245.19922065441563 (T) = (0 -333081.45952622115) / Math.Sqrt((2470054.6598366387 / (299)) + (431462.9967295665 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.12703970946450527 = (381553.9642953301 - 333081.45952622115) / 381553.9642953301 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress_WithState(level: Optimal, file: "sum")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.99311637720159 < 124.82119000093788.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 212.17936835806464 (T) = (0 -107596.0890478904) / Math.Sqrt((278375.45030756196 / (299)) + (168042.16812680333 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.183187568275238 = (131726.8014894105 - 107596.0890478904) / 131726.8014894105 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress_WithoutState(level: Optimal, file: "TestDocument.pdf")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 289.05534928902114 < 661.0085065698161.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 1787.0754419676566 (T) = (0 -287866.1907269797) / Math.Sqrt((3689498.8680765685 / (299)) + (561364.4684397704 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.5870462065581941 = (697090.5590374392 - 287866.1907269797) / 697090.5590374392 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Compression.Brotli.Decompress(level: Optimal, file: "sum")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.4498017857143 < 128.99704417642144.
IsChangePoint: Marked as a change because one of 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 226.35280651621613 (T) = (0 -111697.91531157123) / Math.Sqrt((278020.98396199953 / (299)) + (149536.19699147914 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.179225348265474 = (136088.4075495294 - 111697.91531157123) / 136088.4075495294 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	68e13b68bc20a36a0cfd715d88cf04e4fb9b510b
Compare	00d9488a70ad6e4e8d7aa6e0325d0deba0a14957
Diff	Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IsMatch - Duration of single invocation	226.49 ns	197.46 ns	0.87	0.06	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 197.45699035693306 < 213.81018848157896.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 7/15/2022 5:00:56 PM, 7/18/2022 2:17:16 AM falls between 7/8/2022 10:31:34 AM and 7/18/2022 2:17:16 AM.
IsImprovementStdDev: Marked as improvement because 40.791565560819734 (T) = (0 -197.62898479298354) / Math.Sqrt((279.93746019201313 / (299)) + (1.7870661758892663 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1755274240165288 = (239.7035275033157 - 197.62898479298354) / 239.7035275033157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository