Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LinqBenchmarks.Where00ForX has regressed on x86 #67968

Closed
adamsitnik opened this issue Apr 13, 2022 · 3 comments
Closed

LinqBenchmarks.Where00ForX has regressed on x86 #67968

adamsitnik opened this issue Apr 13, 2022 · 3 comments
Labels
arch-x86 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI tenet-performance Performance related issue
Milestone
7.0.0

Comments

@adamsitnik
Copy link
Member

LinqBenchmarks.Where00ForX has regressed on x86:

Reporting System:

image

Looking at the commit diff, there is nothing suspicious. @AndyAyersMS is there any chance it could be caused by #66618?

Repro:

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_monthly.py net7.0-preview2 net7.0-preview3 --filter LinqBenchmarks.Where00ForX --architecture x86

LinqBenchmarks.Where00ForX

Result Base Diff Ratio Operating System Bit Processor Name
Same 794329808.00 794899693.00 1.00 alpine 3.12 Arm64 Unknown processor
Faster 324113900.00 259144600.00 1.25 debian 11 Arm64 Unknown processor
Same 781263343.50 777613883.00 1.00 ubuntu 18.04 Arm64 Unknown processor
Faster 322527300.00 263559000.00 1.22 ubuntu 20.04 Arm64 Unknown processor
Faster 300441600.00 230009600.00 1.31 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Faster 298900600.00 234569400.00 1.27 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 171170300.00 251297800.00 0.68 Windows 11 Arm64 Unknown processor
Same 200142187.50 180847416.00 1.11 macOS Monterey 12.3 Arm64 Apple M1 Max
Same 171841500.00 161705425.00 1.06 Windows 10 X64 Intel Xeon Platinum 8272CL CPU 2.60GHz
Faster 161237800.00 142624500.00 1.13 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Faster 137784500.00 122582150.00 1.12 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Faster 105536800.00 89634450.00 1.18 Windows 11 X64 AMD Ryzen 9 5900X
Same 229482000.00 255298300.00 0.90 Windows 11 X64 Intel Core i5-4300U CPU 1.90GHz (Haswell)
Slower 185270700.00 322588350.00 0.57 Windows 11 X64 Unknown processor
Faster 124797850.00 105598550.00 1.18 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 144703400.00 131366900.00 1.10 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
Same 170150150.00 179918000.00 0.95 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower 401932143.00 468423204.00 0.86 centos 7 X64 Intel Xeon CPU E5530 2.40GHz
Slower 164042650.00 194109200.00 0.85 debian 11 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 177471100.00 183275700.00 0.97 opensuse-leap 15.3 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 234700149.50 217929681.00 1.08 pop 20.04 X64 Intel Core i7-6600U CPU 2.60GHz (Skylake)
Same 171085614.00 184165107.00 0.93 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 318531056.00 308674736.00 1.03 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same 293873600.00 287123200.00 1.02 ubuntu 20.04 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 297494100.00 273657300.00 1.09 Windows 10 Arm Microsoft SQ1 3.0 GHz
Faster 1059013097.00 883856253.00 1.20 ubuntu 18.04 Arm ARMv7 Processor rev 3 (v7l)
Slower 178325400.00 274649700.00 0.65 Windows 10 X86 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 141388800.00 163779700.00 0.86 Windows 11 X86 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 329635250.00 726786000.00 0.45 Windows 11 X86 Intel Core i7-10510U CPU 1.80GHz
Slower 139309700.00 236116000.00 0.59 Windows 7 SP1 X86 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 288815214.50 318472008.00 0.91 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Same 286576674.50 268661508.00 1.07 macOS Monterey 12.3.1 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Apr 13, 2022
@ghost
Copy link

ghost commented Apr 13, 2022

Tagging subscribers to this area: @dotnet/area-system-linq
See info in area-owners.md if you want to be subscribed.

Issue Details

LinqBenchmarks.Where00ForX has regressed on x86:

Reporting System:

image

Looking at the commit diff, there is nothing suspicious. @AndyAyersMS is there any chance it could be caused by #66618?

Repro:

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_monthly.py net7.0-preview2 net7.0-preview3 --filter LinqBenchmarks.Where00ForX --architecture x86

LinqBenchmarks.Where00ForX

Result Base Diff Ratio Operating System Bit Processor Name
Same 794329808.00 794899693.00 1.00 alpine 3.12 Arm64 Unknown processor
Faster 324113900.00 259144600.00 1.25 debian 11 Arm64 Unknown processor
Same 781263343.50 777613883.00 1.00 ubuntu 18.04 Arm64 Unknown processor
Faster 322527300.00 263559000.00 1.22 ubuntu 20.04 Arm64 Unknown processor
Faster 300441600.00 230009600.00 1.31 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Faster 298900600.00 234569400.00 1.27 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 171170300.00 251297800.00 0.68 Windows 11 Arm64 Unknown processor
Same 200142187.50 180847416.00 1.11 macOS Monterey 12.3 Arm64 Apple M1 Max
Same 171841500.00 161705425.00 1.06 Windows 10 X64 Intel Xeon Platinum 8272CL CPU 2.60GHz
Faster 161237800.00 142624500.00 1.13 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Faster 137784500.00 122582150.00 1.12 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Faster 105536800.00 89634450.00 1.18 Windows 11 X64 AMD Ryzen 9 5900X
Same 229482000.00 255298300.00 0.90 Windows 11 X64 Intel Core i5-4300U CPU 1.90GHz (Haswell)
Slower 185270700.00 322588350.00 0.57 Windows 11 X64 Unknown processor
Faster 124797850.00 105598550.00 1.18 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 144703400.00 131366900.00 1.10 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
Same 170150150.00 179918000.00 0.95 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower 401932143.00 468423204.00 0.86 centos 7 X64 Intel Xeon CPU E5530 2.40GHz
Slower 164042650.00 194109200.00 0.85 debian 11 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 177471100.00 183275700.00 0.97 opensuse-leap 15.3 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 234700149.50 217929681.00 1.08 pop 20.04 X64 Intel Core i7-6600U CPU 2.60GHz (Skylake)
Same 171085614.00 184165107.00 0.93 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 318531056.00 308674736.00 1.03 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same 293873600.00 287123200.00 1.02 ubuntu 20.04 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 297494100.00 273657300.00 1.09 Windows 10 Arm Microsoft SQ1 3.0 GHz
Faster 1059013097.00 883856253.00 1.20 ubuntu 18.04 Arm ARMv7 Processor rev 3 (v7l)
Slower 178325400.00 274649700.00 0.65 Windows 10 X86 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 141388800.00 163779700.00 0.86 Windows 11 X86 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 329635250.00 726786000.00 0.45 Windows 11 X86 Intel Core i7-10510U CPU 1.80GHz
Slower 139309700.00 236116000.00 0.59 Windows 7 SP1 X86 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 288815214.50 318472008.00 0.91 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Same 286576674.50 268661508.00 1.07 macOS Monterey 12.3.1 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)
Author: adamsitnik
Assignees: -
Labels:

arch-x86, area-System.Linq, tenet-performance
Milestone: -

@adamsitnik adamsitnik mentioned this issue Apr 13, 2022
Closed
18 tasks
@eiriktsarpalis eiriktsarpalis removed the untriaged New issue has not been triaged by the area owner label Apr 14, 2022
@eiriktsarpalis eiriktsarpalis added this to the 7.0.0 milestone Apr 14, 2022
@eiriktsarpalis eiriktsarpalis added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed area-System.Linq labels Aug 5, 2022
@ghost
Copy link

ghost commented Aug 5, 2022

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

LinqBenchmarks.Where00ForX has regressed on x86:

Reporting System:

image

Looking at the commit diff, there is nothing suspicious. @AndyAyersMS is there any chance it could be caused by #66618?

Repro:

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_monthly.py net7.0-preview2 net7.0-preview3 --filter LinqBenchmarks.Where00ForX --architecture x86

LinqBenchmarks.Where00ForX

Result Base Diff Ratio Operating System Bit Processor Name
Same 794329808.00 794899693.00 1.00 alpine 3.12 Arm64 Unknown processor
Faster 324113900.00 259144600.00 1.25 debian 11 Arm64 Unknown processor
Same 781263343.50 777613883.00 1.00 ubuntu 18.04 Arm64 Unknown processor
Faster 322527300.00 263559000.00 1.22 ubuntu 20.04 Arm64 Unknown processor
Faster 300441600.00 230009600.00 1.31 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Faster 298900600.00 234569400.00 1.27 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 171170300.00 251297800.00 0.68 Windows 11 Arm64 Unknown processor
Same 200142187.50 180847416.00 1.11 macOS Monterey 12.3 Arm64 Apple M1 Max
Same 171841500.00 161705425.00 1.06 Windows 10 X64 Intel Xeon Platinum 8272CL CPU 2.60GHz
Faster 161237800.00 142624500.00 1.13 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Faster 137784500.00 122582150.00 1.12 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Faster 105536800.00 89634450.00 1.18 Windows 11 X64 AMD Ryzen 9 5900X
Same 229482000.00 255298300.00 0.90 Windows 11 X64 Intel Core i5-4300U CPU 1.90GHz (Haswell)
Slower 185270700.00 322588350.00 0.57 Windows 11 X64 Unknown processor
Faster 124797850.00 105598550.00 1.18 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 144703400.00 131366900.00 1.10 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
Same 170150150.00 179918000.00 0.95 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower 401932143.00 468423204.00 0.86 centos 7 X64 Intel Xeon CPU E5530 2.40GHz
Slower 164042650.00 194109200.00 0.85 debian 11 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 177471100.00 183275700.00 0.97 opensuse-leap 15.3 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 234700149.50 217929681.00 1.08 pop 20.04 X64 Intel Core i7-6600U CPU 2.60GHz (Skylake)
Same 171085614.00 184165107.00 0.93 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 318531056.00 308674736.00 1.03 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same 293873600.00 287123200.00 1.02 ubuntu 20.04 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same 297494100.00 273657300.00 1.09 Windows 10 Arm Microsoft SQ1 3.0 GHz
Faster 1059013097.00 883856253.00 1.20 ubuntu 18.04 Arm ARMv7 Processor rev 3 (v7l)
Slower 178325400.00 274649700.00 0.65 Windows 10 X86 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 141388800.00 163779700.00 0.86 Windows 11 X86 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 329635250.00 726786000.00 0.45 Windows 11 X86 Intel Core i7-10510U CPU 1.80GHz
Slower 139309700.00 236116000.00 0.59 Windows 7 SP1 X86 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 288815214.50 318472008.00 0.91 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Same 286576674.50 268661508.00 1.07 macOS Monterey 12.3.1 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)
Author: adamsitnik
Assignees: -
Labels:

arch-x86, tenet-performance, area-CodeGen-coreclr
Milestone: 7.0.0

@AndyAyersMS
Copy link
Member

Yes this was from #66618. It was fixed by #71659.

newplot - 2022-08-06T094948 243

@ghost ghost locked as resolved and limited conversation to collaborators Sep 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-x86 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI tenet-performance Performance related issue
Projects
None yet
Milestone
7.0.0
Development

No branches or pull requests
3 participants
@eiriktsarpalis @adamsitnik @AndyAyersMS