[Perf -53%] System.Numerics.Tests.Perf_Vector4.GetHashCodeBenchmark

[DrewScoggins]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Changes	diff

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark	Baseline	Test	Test/Base	Modality	Baseline Outlier
GetHashCodeBenchmark	3.13 ns	4.80 ns	1.53	Bimodal	True

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Numerics.Tests.Perf_Vector4*';

Histogram

System.Numerics.Tests.Perf_Vector4.GetHashCodeBenchmark

[2.799 ; 3.009) | @
[3.009 ; 3.320) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[3.320 ; 3.630) | 
[3.630 ; 3.941) | 
[3.941 ; 4.252) | 
[4.252 ; 4.483) | 
[4.483 ; 4.675) | @
[4.675 ; 4.986) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[4.986 ; 5.373) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[5.373 ; 5.771) | @

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[ghost]

Tagging subscribers to this area: @tannergooding, @pgovind
Notify danmosemsft if you want to be subscribed.

[DrewScoggins]

The earliest build we have in that graph is here, https://github.com/dotnet/core-setup/tree/b903a6e815dbcec7acfeef39db1e9189fef7ed69. That is the link to the core-setup that went into the version of core-sdk that we were running. I am not sure if that is before or after had branched for 3.1, but that should at least give a place to start looking for what caused this.

[tannergooding]

There haven't been any notable changes to GetHashCode outside @stephentoub switching it off HashHelpers.Combine back in September: https://github.com/dotnet/runtime/commits/master/src/libraries/System.Numerics.Vectors/src (which is still the current implementation)

• Noting after October the filles moved into S.P.Corelib: https://github.com/dotnet/runtime/commits/master/src/libraries/System.Private.CoreLib/src/System/Numerics

[tannergooding]

We are also talking just shy of 2ns on something that is likely never used (and shouldn't be used) in a HashTable in the first place

[DrewScoggins]

Yeah, it also looks like in #39035 @stephentoub mentioned that this was by design.

I misread that other issue.

[adamsitnik]

Confirmed, 5.0 milestone added:

System.Numerics.Tests.Perf_Vector4.GetHashCodeBenchmark

Conclusion	Base	Diff	Base/Diff	Modality	Operating System	Bit	Processor Name	Base Runtime	Diff Runtime
Slower	2.84	9.76	0.29		Windows 10.0.18363.1016	Arm	Microsoft SQ1 3.0 GHz	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Slower	2.94	4.58	0.64		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20404.3
Slower	3.25	4.37	0.74		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20418.3
Slower	3.80	5.39	0.70		Windows 10.0.19041.450	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Slower	3.04	4.84	0.63		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20403.23
Slower	3.50	5.49	0.64		macOS Mojave 10.14.5	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20404.2

[adamsitnik]

We are also talking just shy of 2ns on something that is likely never used

I am sorry, I've missed this comment. Removing the 5.0 milestone

[jeffhandley]

Closing this as an expected (and acceptable) regression due to dotnet/corefx#40935.