Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize BitOperations uses in CoreLib #35650

Merged
merged 5 commits into from
May 23, 2020
Merged

Optimize BitOperations uses in CoreLib #35650

merged 5 commits into from
May 23, 2020

Conversation

saucecontrol
Copy link
Member

@saucecontrol saucecontrol commented Apr 30, 2020

Following up on #35598, this updates uses of Lzcnt.LeadingZeroCount and Bmi1.TrailingZeroCount to use the equivalent BitOperations methods, which are now optimized for all xarch and for ARM64.

I also found that most uses of LeadingZeroCount were actually calculating log2. This presents opportunities to both simplify the code and to make it run faster on processors without LZCNT support as well as in R2R CoreLib.

When using LZCNT to calculate Log2, the result must be switched to the position of the highest non-zero bit. Since the LeadingZeroCount BSR and software fallbacks calculate log2 first then switch it to LZCNT, any place LZCNT is used where log2 is intended means that the fallback result is double-swapped. The changes here eliminate that.

In switching existing logic over to BitOperations.Log2, I found that most uses were protected against 0 inputs, making the zero-check branch in Log2 unnecessary. I have added an internal Log2Unsafe that skips that protection when the inputs are known to be non-zero. I also made the public Log2 branch-free, by simply setting the low bit of the input. Log2(1) is 0, and log2 of any odd number would round down anyway.

The new Log2 shows a decent performance gain over the previous improvement in #34550

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18363.778 (1909/November2018Update/19H2)
Intel Core i7-6700K CPU 4.00GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=3.1.100
  [Host]     : .NET Core 5.0.0 (CoreCLR 5.0.20.22309, CoreFX 5.0.20.22309), X64 RyuJIT
  Job-TSFLZN : .NET Core 5.0 (CoreCLR 42.42.42.42424, CoreFX 42.42.42.42424), X64 RyuJIT
  Job-BBSNWR : .NET Core 5.0 (CoreCLR 42.42.42.42424, CoreFX 42.42.42.42424), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable  IterationTime=250.0000 ms  
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1  
Method Job Toolchain Mean Error StdDev Median Min Max Ratio Gen 0 Gen 1 Gen 2 Allocated
Log2_uint Job-TSFLZN \corerun-bitops\corerun.exe 599.7 ns 2.10 ns 1.87 ns 599.7 ns 596.6 ns 603.6 ns 0.82 - - - -
Log2_uint Job-BBSNWR \corerun-master\corerun.exe 735.4 ns 1.58 ns 1.32 ns 735.5 ns 733.2 ns 738.2 ns 1.00 - - - -
Log2_ulong Job-TSFLZN \corerun-bitops\corerun.exe 599.8 ns 2.28 ns 2.02 ns 599.5 ns 597.3 ns 604.6 ns 0.81 - - - -
Log2_ulong Job-BBSNWR \corerun-master\corerun.exe 737.8 ns 3.53 ns 2.95 ns 738.3 ns 730.7 ns 742.5 ns 1.00 - - - -

With LZCNT disabled, the improvement is even greater since BSR returns log2 directly. On Intel machines, the fallback is actually faster than the LZCNT version (not true for AMD unfortunately).

Method Job Toolchain Mean Error StdDev Median Min Max Ratio Gen 0 Gen 1 Gen 2 Allocated
Log2_uint Job-FVXROL \corerun-bitops\corerun.exe 555.7 ns 0.71 ns 0.59 ns 555.6 ns 554.8 ns 556.6 ns 0.62 - - - -
Log2_uint Job-JLEBRF \corerun-master\corerun.exe 893.4 ns 2.28 ns 2.13 ns 893.3 ns 889.1 ns 897.6 ns 1.00 - - - -
Log2_ulong Job-FVXROL \corerun-bitops\corerun.exe 555.2 ns 1.19 ns 1.05 ns 555.3 ns 552.7 ns 557.0 ns 0.62 - - - -
Log2_ulong Job-JLEBRF \corerun-master\corerun.exe 892.6 ns 4.36 ns 4.07 ns 892.5 ns 884.8 ns 899.5 ns 1.00 - - - -

{
if (Bmi1.X64.IsSupported)
{
return (int)(Bmi1.X64.TrailingZeroCount(match) >> 3);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code was dead as of #32371, since it was used only on the Vector<byte> fallback for when SSE2 is not available. Since SSE2 is required for BMI1, this was unreachable.

// Flag least significant power of two bit
ulong powerOfTwoFlag = match ^ (match - 1);
// Shift all powers of two into the high byte and extract
return (int)((powerOfTwoFlag * XorPowerOfTwoToHighByte) >> 57);
Copy link
Member Author

@saucecontrol saucecontrol Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BitOperations.TrailingZeroCount(ulong) is now hardware accelerated on all x64 and ARM64. On x86, the value is decomposed and accelerated as a 32-bit call, leaving ARM32 as the only candidate for this software fallback. Since it's doing 64-bit math, the decomposed LUT fallback in BitOperations is probably cheaper.

// TODO: Arm variants
if (Bmi1.X64.IsSupported)
{
return (int)(Bmi1.X64.TrailingZeroCount(match) >> 4);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is still reachable, but the fallback note from above applies.

// if we're able to emit a real tzcnt instruction; the software fallback used by BitOperations
// is too slow for our purposes since we can provide our own faster, specialized software fallback.

if (Bmi1.IsSupported)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to preserve the intent of this one, which was that hardware accelerated tzcnt should be used on little-endian. I can't think of any LE case that wouldn't be better off using BitOperations now, but I might have missed something.
cc @GrabYourPitchforks

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the heads-up! :)
I'm not familiar enough with the larger change but this part LGTM.

@@ -410,18 +410,18 @@ private static unsafe nuint GetIndexOfFirstNonAsciiByte_Sse2(byte* pBuffer, nuin

if ((bufferLength & 8) != 0)
{
if (Bmi1.X64.IsSupported)
if (UIntPtr.Size == sizeof(ulong))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, this path should be faster on all supported 64-bit platforms, but I wasn't sure how that's best expressed here. Should this use conditional compilation, or an explicit if (X86Base.X64.IsSupported || ArmBase.Arm64.IsSupported)?

Copy link
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@jkotas jkotas merged commit ba728d9 into dotnet:master May 23, 2020
@saucecontrol saucecontrol deleted the opt-bitops branch May 27, 2020 00:15
jashook pushed a commit that referenced this pull request May 29, 2020
* Move Mobile test runners to libraries/common and use P2P references (#36411)

* Move Mobile test runners to libraries/common and use P2P references

* PR Feedback

Co-authored-by: Alexander Köplinger <[email protected]>

Co-authored-by: Alexander Köplinger <[email protected]>

* Fix execution of large version bubble composite images (#36373)

Large-bubble composite images are special in having more entries
in the manifest metadata than in the component assembly table:
When the build starts, all component assemblies get hard-injected
into the manifest metadata and subsequently we lazily add those
additional reference assemblies (within the same version bubble)
as we need for encoding signatures.

Thanks

Tomas

* Remove Microsoft.NETCore.Platforms.Future project (#36407)

It's no longer used.

* [master] Update dependencies from mono/linker dotnet/llvm-project dotnet/xharness (#36336)

* Update dependencies from https://github.com/mono/linker build 20200512.1

- Microsoft.NET.ILLink.Tasks: 5.0.0-preview.3.20261.2 -> 5.0.0-preview.3.20262.1

* Update dependencies from https://github.com/dotnet/llvm-project build 20200512.1

- runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: 6.0.1-alpha.1.20261.3 -> 9.0.1-alpha.1.20262.1
- runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: 6.0.1-alpha.1.20261.3 -> 9.0.1-alpha.1.20262.1
- runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: 6.0.1-alpha.1.20261.3 -> 9.0.1-alpha.1.20262.1
- runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools: 6.0.1-alpha.1.20261.3 -> 9.0.1-alpha.1.20262.1
- runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: 6.0.1-alpha.1.20261.3 -> 9.0.1-alpha.1.20262.1
- runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk: 6.0.1-alpha.1.20261.3 -> 9.0.1-alpha.1.20262.1

* Update dependencies from https://github.com/dotnet/xharness build 20200513.4

- Microsoft.DotNet.XHarness.Tests.Runners: 1.0.0-prerelease.20261.4 -> 1.0.0-prerelease.20263.4

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Rework rejit test to ensure subprocess environment is suitable (#36420)

This test is optimization sensitive, but it invokes a subprocess that
will inherit environment variables like `COMPlus_JITMinOpts` that can impact
optimization of code jitted in the subprocess and cause the test to fail.

So, update the parent process code to override `COMPlus_JITMinOpts` and
`COMPlus_JitStress` for the child process.

Closes #35742.

* Implement GetEntrypointExecutableAbsPath on SunOS (#36430)

* Implement `GetEntrypointExecutableAbsolutePath`.
* Fix a warning from newer gawk (v5.0.1 from 2019):
  > ```sh
  > awk: /runtime/src/coreclr/src/nativeresources/processrc.awk:54:
  > warning: regexp escape sequence `\"' is not a known regexp operator
  > ```

* Fix mono runtime build warnings when building iOS config (#36435)

```
  /Users/alexander/dev/runtime/src/mono/mono/mini/aot-runtime.c:5647:13: warning: unused variable 'image' [-Wunused-variable]

  /Users/alexander/dev/runtime/src/mono/mono/mini/simd-intrinsics-netcore.c:11:1: warning: no previous prototype for function 'mono_simd_intrinsics_init' [-Wmissing-prototypes]

  /Users/alexander/dev/runtime/src/mono/mono/utils/mono-state.c:1230:1: warning: no previous prototype for function 'mono_crash_save_failfast_msg' [-Wmissing-prototypes]
  /Users/alexander/dev/runtime/src/mono/mono/utils/mono-state.c:1236:1: warning: no previous prototype for function 'mono_crash_get_failfast_msg' [-Wmissing-prototypes]
```

* Update windows prerequisites to be more specific about SDK (#36438)

* Update windows prerequisites to be more specific about SDK

* Global installation of nightly SDK is required to normally browse the solution files in VS, as there is no way to supply SDKs when the solution files are opened through VS.

* Update windows prerequisites

* allow newer versions of VS workloads

* [debugger] Removing some asserts (#36234)

Removing some asserts and returning err_invalid_argument with an error message when it's possible.

Fixes https://github.com/mono/mono/issues/19651

Co-authored-by: thaystg <[email protected]>

* Work around -no_weak_imports issue with recent Xcode (#36436)

See https://github.com/mono/mono/issues/19393.

We can use the `-Werror=partial-availability` as a good alternative until the Xcode bug is fixed.

* Remove duplicate configuration properties (#36442)

* Update message for generic type. (#36434)

* Consolidate subset projects into ProjectToBuild (#36441)

Consolidating subset projects into a single ProjectToBuild item type to
allow specifying projects to build from different subsets after the
subset was already built.

* [master] Update dependencies from dotnet/arcade mono/linker dotnet/xharness (#36445)

* Update dependencies from https://github.com/dotnet/arcade build 20200511.9

Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.ApiCompat , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.GenAPI , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.GenFacades
 From Version 5.0.0-beta.20258.8 -> To Version 5.0.0-beta.20261.9

* Update dependencies from https://github.com/mono/linker build 20200514.1

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20262.1 -> To Version 5.0.0-preview.3.20264.1

* Update dependencies from https://github.com/dotnet/xharness build 20200514.1

Microsoft.DotNet.XHarness.Tests.Runners
 From Version 1.0.0-prerelease.20263.4 -> To Version 1.0.0-prerelease.20264.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Support additional friendly names on ECC OIDs

Expand the name/OID table to support a m:m relationship (secp256r1 and nistP256 are both 1.2.840.10045.3.1.7; 1.3.14.7.2.3.1 and 1.2.840.113549.1.1.4 are both md5RSA) and add in the alternative names for the secp{size}r1 curves (size in 256, 384, 521).

* Add an atexit handler to bypass calls into ERR_ string routines.

This works around Ubuntu's apparent lack of NO_ATEXIT support in their
build of OpenSSL.

* [mono] Shrink Android APK size (#36437)

* Shrink Android apk size

* bump xharness cli, use cmake config

* Update Linux docker instructions (#36370)

* Update linux-instructions.md to use root build.sh

* Make depproj intermediate output paths unique again (#36451)

Fixes https://github.com/dotnet/runtime/issues/36255

Depprojs depend on IntermediateOutputPath being set in a props file early enough as restore is happening per configuration. Even though it isn't recommended that the TargetFramework property is read before the project is loaded, we currently encode the TargetFramework in the IntermediateOutputPath for depproj files. The long-term fix is to get rid of per configuration restores by getting rid of our depproj files.

* [Mono] Don't set thread name of main thread on Linux (#36116)

* [mono] Record MonoNativeThreadId of the main thread

We would like to know the MonoNativeThreadId (pthread_t on Linux) of the main
thread of the application.  We can identify the main thread (on linux) because
it is the one for which `gettid () == getpid ()`.  (`gettid()` returns a
`pid_t` which is not the same thing as a `pthread_t`, hence this roundabout way
of detecting it.)

A complication arises in embedding scenarios: the main thread is not
necessarily the one that calls `mono_jit_init` or otherwise interacts with the
runtime.  Therefore we do the `gettid() == getpid ()` test at `MonoThreadInfo`
creation time when we call `register_thread`.

If the main thread never interacts with Mono, the main thread is not known to
us.

* [mono] Don't set name of main thread on Linux

Setting the name of the main thread also changes the name of the process.

Fixes https://github.com/dotnet/runtime/issues/35908

The corresponding fix for CoreCLR is https://github.com/dotnet/runtime/pull/34064

* Re-enable test from https://github.com/dotnet/runtime/pull/34064

* Startup optimization w.r.t. manifest access in composite mode (#36446)

During my work on fixing runtime crashes in composite build
with large version bubble enabled I noticed room for startup
perf improvement and a very slight working set optimization:

For component assemblies of a composite image, we can basically
share the cache of those manifest assembly references that
have already been resolved (GetNativeMetadataAssemblyRefFromCache)
within the native image because that is the logical owner
of the manifest metadata.

In the "asymptotic" case of composite images with many
components, the pre-existing behavior was basically
a quadratic O(n^2) algorithm in the number of component
assemblies. This change reduces it to linear in the sense
that all assembly references from the composite image get
resolved only once.

Thanks

Tomas

* Replace System.Native call in Interop.Errors.cs

* Optimize ToScalar() and GetElement() to use arm64 intrinsic (#36156)

* ARM64 intrisic for ToScalar() and GetElement()

* Fixed GetElement to just operate on constants

* Fix bug in rationalize for Vector64<long>

* fix NotSupported issue for GetElement and ToScalar

* Reuse the baseType/retType in impSpecialIntrinsic and impBaseIntrinsic

* Update comment

* fix breaks

* add comments

* ran jit-format

* Refactored to move common logic inside isSupportedBaseType

* review comments

* reuse simdSize

* formatting

* one missing formatting

* Next round of Struct Improvements (#36146)

- Allow accessing a SIMD12 as 16 bytes if it's the single field of a parent struct of 16 bytes.
- On x64/ux don't copy an argument just because it is promoted; the copy would force it to memory anyway.
- Use block init for a promoted struct that's been marked lvDoNotEnregister.
- Allow field-by-field copy of structs with the same fields

* Issue 36212: Remove space in parameter name (#36439)

* Issue 36212: Remove space in parameter name

* Update src/coreclr/src/vm/comutilnative.cpp

Co-authored-by: Stephen Toub <[email protected]>

Co-authored-by: Stephen Toub <[email protected]>

* Remove unnessary guard against switched property names

* R2RTest - Partial composite images (#36471)

Allow compiling composite R2R images which reference assemblies not in the composite. For example, this would allow compiling a set of application assemblies with references to ASP.NET / Framework. Currently R2RTest treats all references as unrooted inputs for the composite image.

* Add issue templates (#36431)

* more

* commented

* yml

* Add an issue template for API proposals

* Update .github/ISSUE_TEMPLATE/api-proposal.md

Co-authored-by: Stephen Toub <[email protected]>

* Update .github/ISSUE_TEMPLATE/api-proposal.md

Co-authored-by: Stephen Toub <[email protected]>

* Update .github/ISSUE_TEMPLATE/api-proposal.md

Co-authored-by: Stephen Toub <[email protected]>

* comment out template instructions

* apply consistent naming

* feedback

* add blank

* H2 and H3

Co-authored-by: Eirik Tsarpalis <[email protected]>
Co-authored-by: Stephen Toub <[email protected]>

* Implement unwrapping a ComWrappers CCW when dumping a stowed exception. (#36360)

* Move mobile test runners to libs.pretest instead of P2P (#36473)

* Add options to ignore default values during serialization (#36322)

* Add options to ignore default values during serialization

* Address review feedback

* Fix typo in test

* Adding the support of reusing machine-wide credentials in the case where the machine is already domain-joined to the LDAP Server. (#36405)

* Adding the support of reusing machine-wide credentials in the case where
the machine is already domain-joined to the LDAP Server.

Co-authored-by: Alexander Chermyanin <[email protected]>

* Addressing feedback and adding ldap4net to TPN

Co-authored-by: Alexander Chermyanin <[email protected]>

* Add dotnet cli to runtime test Helix jobs (#35426)

* Currently managed tools such as crossgen2 and XUnit are run against the built runtime which is slow on Debug builds, and can obscure errors when the XUnit test harness fails due to an introduced runtime bug.
* Add `xunit.console.runtimeconfig.dev.json` which allows `xunit.console` to use the repo-local dotnet.cmd on dev boxes and the same version installed on the path in Helix.
* When running crossgen2 during test execution, support both local dev and Helix scenario when deciding which dotnet to run. On Helix, simply use `dotnet` which assumes a compatible dotnet runtime is in the path. Locally, tests are run with `runtest.cmd|sh` which sets `__TestDotNetCmd` to the repo-local dotnet script. This preserves the characteristic that no machine-wide 5.0 dotnet runtime must be installed for the runtime tests.
* Update batch scripting to also use `dotnet.cmd|sh` when running Crossgen2 replacing corerun.exe.
* crossgen2's `runtimes` folder is not getting copied to `CORE_ROOT` which causes the runtime host to abort the launch on the Unix CI VMs since crossgen.deps.json refers to files in that subfolder. Adjust the `CORE_ROOT` pruning in `Directory.Build.targets` to include subfolders for the two tools that need it.
* Improve XUnit test boilerplate. Printing `Exception.Message` doesn't include stack trace. Use `ToString()` instead.
* Import notargets sdk in `helixpublicwitharcade.proj`. It doesn't use the official sdk so `BundledNETCoreAppPackageVersion` wasn't set. Import the `Microsoft.Build.NoTargets` sdk so we can find the bundled runtime package version.

* Fix DAC layout in checked builds (#35542)

* Improve Uri.Equals by removing unsafe code (#36444)

* Improve Uri.Equals

* Remove not useful and duplicated comments

* Refactor `HasMultiRegRetVal` and `impFixupCallStructReturn`. (#36465)

* Fix target definitions.

They were used in asserts only, no changes.

* Fix failures after a recent HW changes.

* Add a const getter for `ReturnTypeDesc` from a call.

Used to make some new methods const as well.

* Refactor `HasMultiRegRetVal` and `impFixupCallStructReturn`.

Delete an unnecessary nested condition and make checks more straightforward.

* Delete an extra `.` in some dumps.

* Add an additional check that `ReturnTypeDesc` is initialized.

* Remove old `const_cast` around `GetReturnTypeDesc`.

* Replace non-const `GetReturnTypeDesc` with other methods.

* Fix uninitialized `gtSpillFlags, gtOtherRegs, gtReturnTypeDesc` in `fgMorphIntoHelperCall`.

* Implement get_own_executable_path for SunOS (#36514)

* [mono] Improve Android OpenSSL temp hack (#36463)

* [docs] How to run tests on iOS and Android (#36297)

* Update XHarness for latest fixes (#36484)

This includes a couple of fixes in xharness.

* order issue template files (#36524)

* Update 02_api_proposal.md

* Update 02_api_proposal.md

* Consolidate NetCoreAppCurrent properties (#35953)

* Rename 'Blank' issue template to 'Blank issue' (#36533)

* Update and rename 04_blank.md to 04_blank_issue.md

* Use consistent casing

* Ignore .github folder changes in CI (#36530)

* Clean up text in config.yml issue template (#36529)

* Convert Extract(0) to ToScalar() (#36474)

* Convert Extract(0) to ToScalar()
* Update the shim

* Move mobile AppBuilder & AOTCompiler projects into tools-local (#36478)

Moving the projects will make sure their artifacts are always available to the different CI legs.

* Add comment to NetworkStream's ctor about socket.Blocking check (#36539)

* [master] Update dependencies from mono/linker Microsoft/vstest dotnet/xharness (#36525)

* Update dependencies from https://github.com/mono/linker build 20200515.1

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20264.1 -> To Version 5.0.0-preview.3.20265.1

* Update dependencies from https://github.com/microsoft/vstest build 20200515-01

Microsoft.NET.Test.Sdk
 From Version 16.7.0-preview-20200429-01 -> To Version 16.7.0-preview-20200515-01

* Update dependencies from https://github.com/dotnet/xharness build 20200515.1

Microsoft.DotNet.XHarness.Tests.Runners
 From Version 1.0.0-prerelease.20264.9 -> To Version 1.0.0-prerelease.20265.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Viktor Hofer <[email protected]>

* Disable MemoryCacheTest.Trim in arm64 and enable Runtime.Caching tests on Unix (#36494)

* Enable basic generation of ngen pdb from composite image (#36311)

* Enable basic generation of ngen pdb from composite image
- Port non-line number handling portion of ngen pdb from crossgen to r2rdump
  - This pdb generation logic may be used for both composite and non-composite R2R images
  - Only pdb generation is supported. Perfmap generation for unix is not supported
  - pdb generation is only supported on Windows x86 and amd64 platforms. Diasymreader is not supported on other systems
  - Pdb generation does not generation symbols with the same names as crossgen. Instead it uses the name generator from r2rdump. For current needs this should be sufficient
- Update composite file format so that pdb generation process will work. Major difference is that a CorHeader is always produced for composite images now. This CorHeader is not used by the runtime, but is required for DiaSymReader to generate an ngen pdb.

* Next round of multireg preliminary changes (#36155)

This is a zero-diff set of mostly refactoring changes in preparation for supporting multireg locals:
- Move `genRegCopy()` and `genStructReturn()` to codegencommon.cpp, making a new method for `genSIMDSplitReturn` which is target-specific.
- Factor out a new `genUnspillLocal` method from `genUnspillRegIfNeeded()`.
- Similarly factor out `genSpillLocal()`
- Rename `genMultiRegCallStoreToLocal()` and more generally support multireg local stores.
- Fix a bug in the order and shift amount for last-use bits on `GenTreeLclVar`
- Some additional cleanup and preparatory changes

* Fix argument exception warnings on runtime (#35717)

* Fix argument exception warnings on runtime

* Apply feedback

* Addressing feedback and test fix

* Fix test failures

* Fix tests for NetFx run

* Fix suppressed warning for socket.SendAsyn(e) and fix corresponding tests

* Applied feedback

* More HTTP/2 performance (and a few functional) improvements (#36246)

* Use span instead of array for StatusHeaderName

* Fix potential leak into CancellationToken

We need to dispose of the linked token source we create.

Also cleaned up some unnecessarily complicated code nearby.

* Fix HttpConnectionBase.LogExceptions

My previous changes here were flawed for the sync-completing case, and also accidentally introduced a closure.

* Clean up protocol state if/else cascades into switches

* Consolidate a bunch of exception throws into helpers

* Fix cancellation handling of WaitFor100ContinueAsync

* Change AsyncMutex's linked list to be circular

* Remove linked token sources

Rather than creating temporary linked token sources with the request body source and the supplied cancellation token, we can instead just register with the supplied token to cancel the request body source.  This is valid because canceling any part of sending a request cancels any further sending of that request, not just that one constituent operation.

* Avoid registering for linked cancellation until absolutely necessary

We can avoid registering with the cancellation token until after we know that our send is completing asynchronously.

* Remove closure/delegate allocation from WaitForDataAsync

`this` was being closed over accidentally.  I can't wait for static lambdas.

* Avoid a temporary list for storing trailers

Since it only exists to be defensive but we don't expect response.TrailingHeaders to be accessed until after the whole response has been received, we can store the headers into an HttpResponseHeaders instance and swap that instance in at the end.  Best and common case, we avoid the list. Worst and uncommon case, we pay the overhead of the extra HttpResponseHeaders instead of the List.

* Delete dead AcquireWriteLockAsync method

* Reduce header frame overhead

Minor optimizations to improve the asm

* Remove unnecessary throws with GetShutdownException

* Avoid extra lock in SendHeadersAsync

* Move Http2Stream construction out of lock

Makes a significant impact on reducing lock contention.

* Streamline RemoveStream

Including moving credit adjustment out of the lock

* Move response message allocation to ctor

Remove it from within the lock

* Reorder interfaces on Http2Stream

IHttpTrace doesn't need to be prioritized.

* Address PR feedback

* Delete DebugThreadTracking from networking code (#36549)

The System.Net.* libs in dotnet/runtime inherited this from .NET Framework.  To my knowledge it's not once helped flag any issues in dotnet/runtime, it's only built into debug builds, it's become very inconsistent as the code base has evolved, and it's just cluttering stuff up.  So, goodbye.

* Build an apphost with hostfxr and hostpolicy linked in (#36230)

* hostfxr: Build most of hostfxr as a static library

This is part of the work to create an apphost that bundles both hostfxr
and hostpolicy.  The main distinction between the static and shared
versions of hostfxr is that the static version contains a hostpolicy
resolver that references the hostpolicy symbols directly rather than
loading them from a DLL.

* hostpolicy: Build as a static library

This change is part of the work to enable an apphost that bundles both
hostfxr and hostpolicy.  There's no distinction between hostpolicy
that's built as a shared library and as a static library: the shared
library is built by linking an empty object file with the static
library.

* corehost: Allow linking of hostfxr and hostpolicy with apphost

Provide a hostfxr_iface class, that abstracts how the hostfxr functions called
by the early stage in the hosting layer is resolved.

* dotnet: Teach the muxer binary about hostfxr_iface

* apphost: Teach apphost about hostfxr_iface

This provides two implementations of hostfxr_iface: one for the static
apphost, which bundles hostfxr and hostpolicy, and another for the
conventional apphost, which loads them dynamically on startup.

* Add exports for hostfxr and policy

* Exports for unix

* EXPORTS_LINKER_OPTION

* use generateversionscript.awk from ENG

* Move fxr files out of static

* Fixes for Linux

* Fix for   win-x86

* move HEADERS next to SOURCES similarly to other files.

* PR feedback (simplifying hostpolicy_resolver::try_get_dir for static host)

* Publish static_apphost to Microsoft.NETCore.App.Host

* bind to entry points without probing, when in a static host.

* Add a test case

* renamed hostfxr_iface --> hostfxr_resolver_t

* renamed shared --> standalone

* rename static_apphost --> singlefilehost

* Signing exclusions for singlefilehost

* switched StaticHost tst to a different asset (mostly a copy of StandaloneApp)

* get_method_module_path

Co-authored-by: Leandro Pereira <[email protected]>
Co-authored-by: Swaroop Sridhar <[email protected]>

* Produce DropFromSingleFile annotations in RuntimeList.xml (#36578)

* SingleFileHostInclude

* style fixes

* PR feedback

* Fix IL projects build inside visual studio (#36570)

* SslStream.AuthenticateAs sync overloads with SslOptions made public (#36221)

* [mono] Use "dotnet publish" for Android sample with ILLink (#36593)

* Port changes to shared files Nullable.cs, Enum.cs (#36597)

* updating area owners (#36467)

* updating area owners

* adding cross-gen contrib as owner

* Fix StackTraceTests to work with JIT optimization (#36596)

Disable optimization/inlining on methods that are expected to
remain on the stack.

* [mono] Enable some System.Reflection.Emit tests (#35872)

* [interp] Don't share interp_in signatures for different valuetypes (#36520)

We share interp_in wrappers for different types of signatures if the corresponding params are equivalent. This was added in https://github.com/mono/mono/commit/5cbe93884798684efbb81abd79e0e2a170544b75. This was reusing some sharing mechanism used by gsharedvt. Those wrappers are shared with regard to managed->managed transitions so it takes additional freedoms, converting all valuetypes to ValueTuples instances. These can end up being marshalled differently from the initial struct so we can't use this valuetype sharing infrastructure in the interp_in_wrappers which can operate on native structs.

Fixes test_0_marshal_struct_delegate from pinvoke3.cs

Co-authored-by: BrzVlad <[email protected]>

* [master] Update dependencies from mono/linker Microsoft/vstest dotnet/xharness (#36598)

* Update dependencies from https://github.com/mono/linker build 20200515.2

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20265.1 -> To Version 5.0.0-preview.3.20265.2

* Update dependencies from https://github.com/microsoft/vstest build 20200515-03

Microsoft.NET.Test.Sdk
 From Version 16.7.0-preview-20200515-01 -> To Version 16.7.0-preview-20200515-03

* Update dependencies from https://github.com/dotnet/xharness build 20200515.8

Microsoft.DotNet.XHarness.Tests.Runners
 From Version 1.0.0-prerelease.20265.1 -> To Version 1.0.0-prerelease.20265.8

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Removed left-over NUnit references from comments (#36606)

* Sync crossgen2 shared files (#36610)

* CMake: Point download URLs directly to https:// (#36615)

* Naricc/ci interpreter arm64 (#36258)

Change mono interpreter runs to be a scenario instead of a seperate leg. Also enable arm64 interpreter runs, and add test exclusions.

* Single-File: Run from Bundle

This change implements:

* Runtime changes necessary to load assemblies directly from the bundle:
    * Design notes about [Load from Bundle](https://github.com/dotnet/designs/blob/master/accepted/2020/single-file/design.md#peimage-loader)
    * Most of these changes are directly from https://github.com/dotnet/coreclr/pull/26504 and https://github.com/dotnet/coreclr/pull/26904

* Hostpolicy change to not add bundled assemblies to TPA list:
    * Design notes about [Dependency Resolution](https://github.com/dotnet/designs/blob/master/accepted/2020/single-file/design.md#dependency-resolution)
    * TBD (separately) items: Fix for hammer servicing #36031

Fixes #32822

* Address some code review feedback

* Add iOS x86 build (#36602)

Needed for the iOS Simulator for 32bit iOS devices.

* [mono] Fix AssemblyLoadContext.GetRuntimeAssembly() for AssemblyBuilders (#36367)

* Fix mono file name on iOS/Browser (#36646)

We initially intended to just use libmono.so/dylib as the name to simplify and follow the libcoreclr.dylib pattern and we did that by just copying to a different name after the build.

However that didn't work on Android since the name gets embedded inside the binary and Android checks that these match, so we'd either have to change the (auto)make to use the correct library name (and possibly creates complex conditionals in the Makefile for netcore) or go back to using `libmonosgen-2.0` on iOS.

We decided to do the latter.

* [mono] Linking statically ICU shim on mono (#35790)

Linking statically ICU shim on mono for windows, linux, macOs and android.

* Update dependencies from https://github.com/mono/linker build 20200518.2 (#36647)

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20265.2 -> To Version 5.0.0-preview.3.20268.2

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Sync one more shared crossgen2 file (#36638)

* Delete OrderBy(...).First{OrDefault}(...) optimization (#36643)

The optimization removes the O(n log n) cost of the OrderBy. But it can result in executing the predicate passed to First{OrDefault} more than in .NET Framework; it would always execute it n times, whereas previously it would execute it <= n times.  Developers have expressed concern about the change, in particular when using a relatively expensive predicate on a relatively short list, or when unadvisedly relying on side-effecting predicates.

* Fix failing Sockets tests after argument exception changes (#36645)

* Removed redundant visibility check (#36648)

* Remove unnecessary initialization from Utf8JsonWriter ctors (#36651)

* Remove PreserveDependency on non-existing type (#36657)

The DefaultArrayConverter type was been refactored away causing the linker to warn about it.

* Produce Mono+LLVM runtime packs on desktop platforms (#35841)

* Add LLVM Mono runtime build

* Switch from 'llvm' boolean to 'runtimeVariant' freeform string in yaml

This makes it easier to add oddball variant builds, without a big pile of booleans for every possible variant

* Add an LLVM suffix to installer nupkgs

* Add runtimeVariant to CoreCLR artifact names

* Add installer run for LLVM JIT Mono

* Actually specify LLVM or not to installer build

* Unique name for LLVM installer run

* Ensure log uploads are disambiguated

* Fix dependency in full matrix

* Add LLVMAOT variant, which bundles llc/opt for current arch

* Make sure we don't use Mono.LLVM package names on CoreCLR or Mobile

* Fix perf runs to deal with runtimeVariant

* Try to reconcile perf test artifact names

* Make bundling llc/opt the default when LLVM enabled on Mono

* Fix R2RTest parsing of the issues.targets (#36650)

The format of the file has changed a bit some time ago, but the R2RTest
wasn't updated accordingly, so test exclusion stopped working. This
change fixes it.

* Remove BuildOS (#36667)

* Prefix protected fields with underscore for the internal XmlRawWriter (#35759)

* Prefix protected fields with underscore for the internal XmlRawWriter and its descendant classes

* WASM app builder changes. (#36422)

* Add an ExtraAssemblies parameter to the WasmAppBuilder task.

* Pass more assemblies to the pinvoke table generator.

* Improve the wasm sample.

* Move WasmAppBuilder to tools-local.

* [mono] Enable System.Runtime.Tests on Android (#36655)

* Rewrite NegotiateStream.XxAsync operations with async/await (#36583)

* Rewrite NegotiateStream.Read/Write* operations with async/await

Gets rid of a bunch of IAsyncResult cruft and makes the XxAsync APIs cancelable.

* Combine NegoState into NegotiateStream

* Rewrite AuthenticateAs* with async/await

* Add more NegotiateStream tests

Including for cancellation and a product fix to enable cancellation.

* Update ref with overrides

* Remove custom IAsyncResults from System.Net.Security

* Fix UnitTests project

* Libunwind1.5rc2 (#36027)

* Add libunwind 1.5rc2 source

Rename unused autoconfig dir aux -> aux_ consistent with
original checkin

* Delete files added by 1.3-rc1

* Update libunwind version in CMake

* Add libunwind-version.txt

* Add changes for SunOS from @am11

* Revert change to oop

* Remove obsolete add_definition

* Fix musl build

* Fix comment

* Be consistent and use HOST

* Fix error in unw_sigcontext libunwind/libunwind#179

Introduced by libunwind/libunwind#71

__reseverved needs to be big enough to store a unw_fpsimd_context_t
Which includes 32 128-bit registers, stored as 64 64-bit half registers.
Fix off by 2 issue

* Arm64 support !UNWIND_CONTEXT_IS_UCONTEXT_T

Co-authored-by: Adeel <[email protected]>

* Checking strings against length seem to have better perf (#36443)

* Chckecing strings against length seem to have better perf

* Apply suggestions from code review

Co-authored-by: Miha Zupan <[email protected]>
Co-authored-by: Ben Adams <[email protected]>

* Apply suggestions from code review

Co-authored-by: Ben Adams <[email protected]>

* Update src/libraries/System.Drawing.Common/src/System/Drawing/BitmapSelector.cs

Co-authored-by: Ben Adams <[email protected]>

* Apply suggestions from code review

* Update src/libraries/System.Private.CoreLib/src/System/Reflection/AssemblyNameFormatter.cs

* fix build error

* Update SmtpClient.cs

* Update AssemblyNameFormatter.cs

* Update LoggingEventSource.cs

* Fix build error

* Apply suggestions from code review

* Update src/libraries/System.ServiceModel.Syndication/src/System/ServiceModel/Syndication/SyndicationContent.cs

* Fix build error

* Apply suggestions from code review

Co-authored-by: Miha Zupan <[email protected]>
Co-authored-by: Ben Adams <[email protected]>

* Fix RuntimeInformation.IsOSPlatform for Browser/WASM (#36665)

The native code was still using the previous `WEBASSEMBLY` name instead of `BROWSER` as decided in https://github.com/dotnet/runtime/issues/33328.

* [master] Update dependencies from mono/linker Microsoft/vstest (#36692)

* Update dependencies from https://github.com/mono/linker build 20200518.5

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20268.2 -> To Version 5.0.0-preview.3.20268.5

* Update dependencies from https://github.com/microsoft/vstest build 20200518-01

Microsoft.NET.Test.Sdk
 From Version 16.7.0-preview-20200515-03 -> To Version 16.7.0-preview-20200518-01

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Remove duplicated code from SR.cs (#36277)

* Add RequiresUnreferencedCodeAttribute (#36674)

This attribute is used by the linker to know which methods are unsafe to use when an application is trimmed.

Fix #33862

* Revert unintentional alignment change for Vector256<T> (#36673)

Addresses https://github.com/dotnet/runtime/pull/35864#issuecomment-629803015.  In an earlier change I also added the `type.Instantiation[0].IsPrimitive` condition to the `IsVectorType` predicate.  Revert that part as well and allow only primitive numeric types for HFA/HVA purpose.  The same list of 10 types is recognized in `MethodTable::GetVectorSize` and `Compiler::getBaseTypeAndSizeOfSIMDType`.

* Improve sort function in crossgen2 (#36676)

- Increase memory locality substantially in composite images
- Sort non-generic methods with module together
- Sort generic methods near other generic methods with similar instantiations

Json benchmark showed improvement from 237,827 RPS to 270,306 RPS, and reduced working set by about 10MB

* Fixed Incorrectly named function arguments in TryGetPlatformSocketOption. (#36694)

Fixes #36686.

* Try using socket syscalls that accepts a single buffer to improve performance (#36371)

* Try using socket syscalls that accepts a single buffer to improve performance

* Remove ref from Receive calls

* Prefix Pal methods invoking methods with Sys

* Also use single-buffer syscalls for sync Socket Receive methods

* Improve comment

* PR feedback

* Assert SocketAddress null instead of checking

* fix handling of Ssl2 and enable disabled tests (#36098)

* attempt to fix ssl2

* adjust length calculation

* enable tests

* fix ReadAsyncInternal

* use PlatformDetection.SupportsSsl2

* feedback from review

Co-authored-by: Tomas Weinfurt <[email protected]>

* Removed extra slash. (#36722)

Remove extra / from path on runtimetests.

* Contain non-candidate regOptional lclVars (#36601)

* Adding a ci leg for Source build (#36141)

* successfullsource build

* adding a new source build leg.

* remove yy

* add default vlaue

* addressing feedback

* use boolean value

* addind comment and other feedback

* adding colon and removing unintentional change

* adding default value of isSourceBUild

* Remove TheadPool initialization volatile (#36697)

* Remove TheadPool initialization volatile

* Better ThreadPoolGlobals setup for Mono

* Feedback

* Move back to lambda

* Remove duplicated tests from System.Text.Json.Tests (#36483)

* Add build information to libraries helix jobs (#36713)

* Add build information to libraries helix jobs

* Remove BUILD_URI

* Disable EventPipeProvider Context for runtime providers on EventPipe rundown

* JIT: fix no return call accounting (#36719)

The jit tracks the number of no return calls to determine if it should run
throw helper merging and to decide if no return calls should tail called.

The accounting is currently done when the calls are initially imported,
so if code is duplicated (by say finally cloning the count may end up being
an under-estimate. While not a correctness issue, it is better for the count
to be accurate (or an over-estimate).

So, update the count when cloning a no-return call.

Closes #36584.

* Remove Fedora29 from test matrix and add Fedora32 (#36716)

* Code sharing between generic methods in member accessor (#36710)

* Code sharing between generic methods in member accessor

* Removed redudant value type check

* Use latest compiler toolset from Arcade (#36741)

* [mono] Fix iOS sample and use `dotnet publish` (#36745)

Use `dotnet publish` with linker just like for Android.

* [interp] Small cleanups (#36706)

Co-authored-by: BrzVlad <[email protected]>

* [interp] Fix interp entry for methods with lots of arguments in llvmonly+interp mode. (#36678)

Fixes https://github.com/mono/mono/issues/19801.

<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->

Co-authored-by: vargaz <[email protected]>

* Use a Brewfile for installing brew packages (#36747)

This means we won't be upgrading existing packages on the system that we don't need for the build.
Marks install-native-dependencies.sh as executable (+x) so we don't need to start it with `sh` in the build .yml

Fixes https://github.com/dotnet/runtime/issues/36727

* Adding OSX support for System.DirectoryServices.Protocols (#36669)

* Adding OSX support for System.DirectoryServices.Protocols

* Addressing PR Feedback

* Fixing issue in netfx builds where local member is assigned but never used

* Optimize call indirect for R2R, Arm and Arm64 scenarios (#35675)

* Use a different approach to optimize the indirect calls for R2R

During lowering, don't create a controlExpr for indirect call. Instead use
temp register for such calls and during codegen, load the indirect from x11
into that temp register before calling the address in that temp register.

* Add FullAOT mode for Simulator to AppleAppBuilder (#36759)

* Add managed array type support for EventPipe (#36242)

Add support for emitting an event with an arbitrary number of arguments over EventPipe.

* Remove overwriting RuntimeIdentifier in runtime.depproj for wasm/ios/android (#36757)

It shouldn't be needed anymore since we have real assets for these targets now.

* Cache Uri.IdnHost and Uri.PathAndQuery (#36460)

* Cache Uri.IdnHost and Uri.PathAndQuery

* Continue caching DnsSafeHost

* Move PathAndQuery cache from MoreInfo to UriInfo

* Move DnsSafeHost and IdnHost logic to IdnHost getter

* Add more Host tests

* [interp] Add separate opcode for pop vt (#36760)

Before this change, pop-ing a vt from the stack was done in 2 opcodes, a MINT_POP (decrementing the stack) and the weird MINT_VTRESULT (decrementing the vtstack). However, optimizations could have removed both the initial loading of the value type on the stack as well as the MINT_POP, leaving MINT_VTRESULT underflowing the vtstack. Fix this and cleanup the code by pop-ing a value type from the stack in a single instruction.

Co-authored-by: BrzVlad <[email protected]>

* [master] Update dependencies from mono/linker dotnet/llvm-project dotnet/xharness (#36764)

* Update dependencies from https://github.com/dotnet/llvm-project build 20200518.2

runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk
 From Version 9.0.1-alpha.1.20262.1 -> To Version 9.0.1-alpha.1.20268.2

* Update dependencies from https://github.com/mono/linker build 20200519.1

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20268.5 -> To Version 5.0.0-preview.3.20269.1

* Update dependencies from https://github.com/dotnet/xharness build 20200520.1

Microsoft.DotNet.XHarness.Tests.Runners
 From Version 1.0.0-prerelease.20265.8 -> To Version 1.0.0-prerelease.20270.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Support !JitDoOldStructRetyping on  other platforms. (#35943)

* Add more test cases.

* Initialize `ReturnTypeDesc` when we keep struct types.

* Add a few const modifiers.

* Additional checks in `LowerRet`

* Support `return double(cnst int)`.

* Optimize `LowerRetStruct`: no need for bitcast when read from memory.

* Prepare `LowerNode` for store local and local field to multireg.

* Compile the new methods with FEATURE_MULTIREG_RET.

* Improve `LowerRetStructLclVar`.

Don't use bitcast if the source is in memory or has the same type.

* Extract `LowerStoreLocCommon`.

* Support 3, 5. 6, 7 bytes structs in `LowerCallStruct`.

Move call handling to the users.

* Disable `JitDoOldStructRetyping` for x86 and x64.

Windows  x64 was supported in a previous PR, this adds x86 (Windows and Linux) and x64 Unix.

* Fix suggestions.

* Disable by default for the merge.

* [mono] Implement AsAny marshalling for simple arrays (#35686)

* [mono] Implement AsAny marshalling for simple arrays

Simple here means of rank 1, and with a blittable value type as element type.

* Also implement char arrays

* Do not allow in-params to be marshalled this way, as the callee would have write-ability that is not allowed by in-params

* JIT: widen type of constant or special static fields (#36769)

Since we're pushing the value of these fields we need to widen the value
to a stack type.

Addresses issues seen in #36592.

* Optimize ToVector128, ToVector128Unsafe and Vector128.GetLower() (#36732)

* Optimize ToVector128 , ToVector128Unsafe and Vector128.GetLower()

* Fix evaluate paths bug when multiple include paths are specified (#36780)

* Fix signing due to subset projects rename (#36791)

* Adding a case requiring RuntimeJit (#36772)

Handle CORINFO_HELP_THROW_NOT_IMPLEMENTED

* Fix non-ascii absolute file path handling in Uri (#36429)

* Fix non-ascii absolute file path Uri handling

* Fix Unix tests

* Revert unrelated style change

* Add asserts to FilePathHandlesNonAscii to test AbsoluteUri roundtrips properties

* Update public GC.AddMemoryPressure() and GC.RemoveMemoryPressure() (#36767)

* Update public GC.AddMemoryPressure() and GC.RemoveMemoryPressure()
to use newer algorithm.

* Remove old AddMemoryPressure/RemoveMemoryPressure impls

* Update test.

* Removing duplicate tests from coreclr directory.

* Fix two issues with hosting on SunOS (#36527)

Fix two issues with hosting on SunOS:
* Compile corehost without use-cxa-atexit with gcc
* Disable RapidJson's 48-bit optimization on SunOS

* Compile without [`use-cxa-atexit`](https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Dialect-Options.html) in case of gcc
  * libc tries to invoke `pthread_cond_destroy()` in `dlclose()`ed fxr library in the consumer code. PR disables this feature.
  * Backtrace available at http://sprunge.us/bx4dlk. This happens _after_ the main() exits.
* Disable RapidJson's 48-bit pointer optimization on SunOS.
  * This optimization relies on zeros in higher 16-bits, whereas SunOS has 1s. More details at https://github.com/Tencent/rapidjson/issues/1596.
  * The impact here was that `runtimeOptions` key available in `hwapp.runtimeconfig.json` was not located by RapidJson's `FindMember()` API and we were getting `false` from: https://github.com/dotnet/runtime/blob/78b303df8fbb242985d049a277d0d199cafd51b5/src/installer/corehost/cli/runtime_config.cpp#L416-L422

Contributes to: #34944.

* Add R2R testing on Windows arm/arm64 (#36793)

* Add JIT EventCounters (#36489)

* Add JIT counters

* Some renames

* fix build

* change return type of GetMethodsJittedCount to int

* remove ifdef

* Fix

* CR feedback

* More CR feedback

* More CR Feedback

* more code review feedback

* Fix build error and add displayunit to IL bytes jitted counter

* Fix missing symbol issues in SunOS build (#36455)

* Fix missing symbol issues in SunOS build

* Fix SIGSEGV due to invalid free()

* Fixing tests on android. (#36788)

The linker was removing the icu shim functions even exporting them. The unique solution that we found until now is to force the linking using the -u flag.
This is a temporary fix until we don't implement QCalls.
Fixes https://github.com/dotnet/runtime/issues/36685

* Add autoconf, automake and libtool as dependencies (#36475)

* Add autoconf, automake and libtool as dependencies

* Update instructions

* Handle multi-reg copies/reloads (#36802)

Fix #36619

* Adding check to bypass decoding pdbs that have no sequence points. (#36771)

Unity has a pdb with zero sequence points somehow. If a user added a reference to the library and then attempted to set a managed breakpoint mono would hit an assert and crash when it would attempt to decode this pdb.

Call stack would look like:
```
>	mono-2.0-boehm.dll!mono_ppdb_get_seq_points(_MonoDebugMethodInfo * minfo, char * * source_file, _GPtrArray * * source_file_list, int * * source_files, MonoSymSeqPoint * * seq_points, int * n_seq_points) Line 508	C
 	mono-2.0-boehm.dll!mono_debug_get_seq_points(_MonoDebugMethodInfo * minfo, char * * source_file, _GPtrArray * * source_file_list, int * * source_files, MonoSymSeqPoint * * seq_points, int * n_seq_points) Line 1093	C
 	mono-2.0-boehm.dll!get_source_files_for_type(_MonoClass * klass) Line 6920	C
 	mono-2.0-boehm.dll!get_types_for_source_file(void * key, void * value, void * user_data) Line 7003	C
 	mono-2.0-boehm.dll!monoeg_g_hash_table_foreach(_GHashTable * hash, void(*)(void *, void *, void *) func, void * user_data) Line 364	C
 	mono-2.0-boehm.dll!mono_de_foreach_domain(void(*)(void *, void *, void *) func, void * user_data) Line 95	C
 	mono-2.0-boehm.dll!vm_commands(int command, int id, unsigned char * p, unsigned char * end, Buffer * buf) Line 7341	C
 	mono-2.0-boehm.dll!debugger_thread(void * arg) Line 10323	C
 	mono-2.0-boehm.dll!start_wrapper_internal(StartInfo * start_info, unsigned __int64 * stack_ptr) Line 1241	C
 	mono-2.0-boehm.dll!start_wrapper(void * data) Line 1315	C
 	kernel32.dll!00007ffc12017bd4()	Unknown
 	ntdll.dll!00007ffc121ece51()	Unknown
```

Adding a check to ignore pdbs like this prevents the crash and debugging can continue normally.

Related unity issue: https://issuetracker.unity3d.com/issues/macos-editor-crashes-on-mono-log-write-logfile-when-attaching-a-debugger-and-then-setting-a-breakpoint



Co-authored-by: UnityAlex <[email protected]>

* Allow building Browser/wasm config on Windows (#36816)

Seems to work fine for the `Mono.CoreLib` subset, which is all we care about in the IL Linker-land.

* Enable passing coreclr tests (#36702)

* Updating the HWIntrinsics to support dropping unnecessary casts (#36512)

* Updating the HWIntrinsics to support dropping unnecessary casts

* Applying formatting patch

* Only drop the cast operation if the source is greater than or equal to the expected size

* Minor documentation improvements (#36834)

* [Arm64] ASIMD Shift instructions (#36552)

* sqrshl

* sqrshrn

* sqrshrn2

* sqrshrun

* sqrshrun2

* sqshl

* sqshlu

* sqshrn

* sqshrn2

* sqshrun

* sqshrun2

* srshl

* sshl

* uqrshl

* uqrshrn

* uqrshrn2

* uqshl

* uqshrn

* uqshrn2

* urshl

* ushl

* Stop emitting weird intermediate folder artifacts/tests/artifacts (#36833)

The logic in dir.common.props for relativizing test directories
doesn't work well for the generated XUnit wrapper csproj files as
these are generated under artifacts\tests. Relativizing this
directory against src\coreclr\tests\src ended up with a weird
sequence that ended up duplicating the tests/artifacts folder
level. I have fixed this by explicitly passing the root directory
for relativization for the XUnit wrapper projects.

Thanks

Tomas

Fixes: #1655

* Adding A log presence check before trying to delete (#36779)

* checking if log is present before deleteing

* Update src/libraries/System.Diagnostics.EventLog/tests/EventLogTests/EventLogSourceCreationTests.cs

Co-authored-by: Krzysztof Wicher <[email protected]>

Co-authored-by: Krzysztof Wicher <[email protected]>

* Move app local ICU string length validation to native shim (#36735)

* Move app local string length validation to native shim

* Fix build

* JIT: don't call back into morph from fgRemoveConditionalJump (#36792)

This can get called before morph and run into issues where morph
specific data is not yet initialized. The remorphing wasn't going to
do much other than extract side effects and rethread the statement
ordering. So just do these latter bits.

Closes #36468.

* [wasm][debugger] Allow invoking property getters for objects (#35934)

Co-authored-by: radical <[email protected]>

* Remove Microsoft.DotNet.PlatformAbstractions (#36707)

* Remove Microsoft.DotNet.PlatformAbstractions

This library overlapped with other System APIs and is now obsolete. For API replacement:

ApplicationEnvironment.ApplicationBasePath => AppContext.BaseDirectory
HashCodeCombiner => System.HashCode
RuntimeEnvironment.GetRuntimeIdentifier() => RuntimeInformation.RuntimeIdentifier
RuntimeEnvironment.OperatingSystemPlatform => RuntimeInformation.IsOSPlatform(OSPlatform)
RuntimeEnvironment.RuntimeArchitecture => RuntimeInformation.ProcessArchitecture
RuntimeEnvironment.OperatingSystem => RuntimeInformation.OSDescription
RuntimeEnvironment.OperatingSystemVersion => RuntimeInformation.OSDescription / Environment.OSVersion.Version

Fix #3470

* Porting more of the SIMD intrinsics to be implemented as HWIntrinsics (#36579)

* Porting Ceiling and Floor to use SimdAsHWIntrinsic

* Porting SquareRoot to use SimdAsHWIntrinsic

* Porting ConditionalSelect to use SimdAsHWIntrinsic

* Porting get_AllBitsSet, get_Count, and get_Zero to use SimdAsHWIntrinsic

* Porting op_Explicit to use SimdAsHWIntrinsic

* Changing Vector2/3/4 and Vector<T>.Equals to forward to operator ==

* Removing SIMDIntrinsicAbs

* Removing SIMDIntrinsicMax and SIMDIntrinsicMin

* Removing SIMDIntrinsicCeil and SIMDIntrinsicFloor

* Porting op_Equality and op_Inequality to use SimdAsHWIntrinsic

* Removing SIMDIntrinsicSqrt

* Removing SIMDIntrinsicSelect

* Removing SIMDIntrinsicBitwiseAndNot and SIMDIntrinsicBitwiseXor

* Removing SIMDIntrinsicGreaterThanOrEqual and SIMDIntrinsicLessThanOrEqual

* Removing SIMDIntrinsicGreaterThan and SIMDIntrinsicLessThan

* Removing SIMDIntrinsicInstEquals

* Removing SIMDIntrinsicOpEquality and SIMDIntrinsicOpInEquality

* Porting this.Equals to use SimdAsHWIntrinsic

* Don't handle IEquatable`1.Equals via SimdAsHWIntrinsic

* Applying formatting patch

* Account for op2 being able to precede op1 in the LIR order

* Ensure SimdAsHWIntrinsic with 0 args are properly handled

* Fixup the arm64 LowerHWIntrinsicCmpOp implementation to use LowerNodeCC

* Fixing an assert in the NotSupported HWIntrinsic tests

* Use socketpair to implement Process.Start redirection (#34861)

* Use socketpair to implement Process.Start redirection

Today on Unix, we create an anonymous pipe via pipe/pipe2 to be used for stdin/stdout/stderr on processes created by Process.Start.  We then wrap the resulting file descriptors with FileStreams to hand out via Process.StandardInput/Output/Error.  This has a few issues, however. Any async operations on the resulting stream (or wrapping stream reader) will actually be async-over-sync, and that in turn means that a) any async read will end up blocking a thread pool thread until it's satisified, and b) the operation isn't cancelable.  The implications of (b) are obvious, and the problem with (a) is that code which launches a bunch of processes and uses BeginOutput/ErrorReadLine or the like will end up blocking a bunch of thread pool threads.

This change replaces the pipe/pipe2 calls with socketpair calls, and instead of wrapping the resulting file descriptors with FileStream, wraps them in Sockets and NetworkStreams.  This gives us the full capabilities of the networking stack, which fully supports asynchronous and cancelable reads and writes.

* Try to fix macOS failures with socketpair

* Extract shared functionality to a TryRemoveCastIfPresent method (#36837)

* Added CreateDelegate<T> overloads to MethodInfo (#36680)

* Added generic overloads for MethodInfo.CreateDelegate<>()

* Added full support of CreateDelegate<T> and the necessary tests.

* Refactored to apply the new CreateDelegate overloads as necessary throughout runtime.

* Reverted Linq.Expressions changes and fully qualified Delegate to System.Delegate

* [mono] Fix MonoRunner.java - assets were not copied correctly (#36824)

* Move where wasm binaries are generated (#36781)

This change shifts producing wasm binaries to the native dir so that they'll come over to the installer on CI.

* Strip the ILLinkTrim.xml file from System.Security assemblies (#36864)

Contributes to #35199

* Consider armel arch (#36872)

* Fix a summary typo in ConsoleLoggerOptions.cs (#36870)

* Remove x86 OSX code (#36874)

This removes x86 OSX code that was never used for .NET Core.

* Ensure Image.Save can handle non readable / seekable Streams (#36805)

* Ensure Image.Save can handle non readable / seekable Streams

* Address feedback

* Fix Image.Save on Unix for write only non-seekable stream

* Remove test which uses bogus handle value

* Use linux-musl runtime on Alpine Helix workers (#36827)

* Use linux-musl runtime on Alpine Helix workers

* Add OS Subgroup to send-to-helix-step

* Pass in osSubgroup

Also, capitalize it in line with the rest of the repo

* Use __TargetOS which understands linux_musl for rid generation

I don't fully understand why we use `TargetOS` in `helixpublishwitharcade.proj` but set `__TargetOS` in the calling `send-to-helix-step.yml`.

* Fix wasm build when runtime configuration and configuration are different (#36855)

* Fix wasm build when runtime configuration and configuration are different

* PR Feedback

* Fix CoreCLR initialization on Illumos-based distros (#36808)

* [Arm64] Polynomial Multiply Long Intrinsics (#36853)

Implements PolynomialMultiplyWideningLower and PolynomialMultiplyWideningUpper

* Add NotNullIfNotNull, MemberNotNull attributes to doc (#36856)

* [Arm64] Follow-up on renaming intrinsics to Narrowing for consistency with Widening (#36857)

* Rename AddHighNarrow to AddHighNarrowing

* Rename AddRoundedHighNarrow to AddRoundedHighNarrowing

* Rename ExtractAndNarrowHigh to ExtractNarrowingUpper

* Rename ExtractAndNarrowLow to ExtractNarrowingLower

* Rename SubtractHighNarrow to SubtractHighNarrowing 

* Rename SubtractRoundedHighNarrow to SubtractRoundedHighNarrowing

* Renamed pInitRegZeroed to pInitRegModified, initRegZeroed to initRegM… (#36321)

* Add MONO_LOADER_LIBRARY_NAME define (#36835)

* Fixing make dist after implementation of static ICU Shim. (#36795)

Fixing make dist after implementation of static ICU Shim.

Co-authored-by: thaystg <[email protected]>

* Improve obsoletion message for IgnoreNullValues (#36886)

* Disable System.Numerics.Tests.Vector3Tests.Vector3InequalityTest on arm64 (#36906)

* Pipelines performance tweaks (#36815)

* Pipelines performance tweaks

* Packaging

* Don't use Unsafe for array cast

* Remove Unsafe use

* Remove System.Globalization.Native shim from shared framework (#36904)

* Remove System.Globalization.Native shim from shared framework

* Add System.Globalization.Native sources to mono paths in pipeline

* Remove dead Exception handling code in Uri (#36865)

* Make Vector256 dependent on AVX instruction set (#36895)

* [Arm64] ASIMD Shift Intrinsics (#36830)

* ShiftArithmetic

* ShiftArithmeticRounded

* ShiftArithmeticRoundedSaturate

* ShiftArithmeticRoundedSaturateScalar

* ShiftArithmeticRoundedScalar

* ShiftArithmeticSaturate

* ShiftArithmeticSaturateScalar

* ShiftArithmeticScalar

* ShiftLeftLogical

* ShiftLeftLogicalSaturate

* ShiftLeftLogicalSaturateScalar

* ShiftLeftLogicalSaturateUnsigned

* ShiftLeftLogicalSaturateUnsignedScalar

* ShiftLeftLogicalScalar

* ShiftLeftLogicalWideningLower

* ShiftLeftLogicalWideningUpper

* ShiftLogical

* ShiftLogicalRounded

* ShiftLogicalRoundedSaturate

* ShiftLogicalRoundedSaturateScalar

* ShiftLogicalRoundedScalar

* ShiftLogicalSaturate

* ShiftLogicalSaturateScalar

* ShiftLogicalScalar

* ShiftRightArithmetic

* ShiftRightArithmeticAdd

* ShiftRightArithmeticAddScalar

* ShiftRightArithmeticNarrowingSaturateLower

* ShiftRightArithmeticNarrowingSaturateUnsignedLower

* ShiftRightArithmeticNarrowingSaturateUnsignedUpper

* ShiftRightArithmeticNarrowingSaturateUpper

* ShiftRightArithmeticRounded

* ShiftRightArithmeticRoundedAdd

* ShiftRightArithmeticRoundedAddScalar

* ShiftRightArithmeticRoundedNarrowingSaturateLower

* ShiftRightArithmeticRoundedNarrowingSaturateUnsignedLower

* ShiftRightArithmeticRoundedNarrowingSaturateUnsignedUpper

* ShiftRightArithmeticRoundedNarrowingSaturateUpper

* ShiftRightArithmeticRoundedScalar

* ShiftRightArithmeticScalar

* ShiftRightLogical

* ShiftRightLogicalAdd

* ShiftRightLogicalAddScalar

* ShiftRightLogicalNarrowingLower

* ShiftRightLogicalNarrowingSaturateLower

* ShiftRightLogicalNarrowingSaturateUpper

* ShiftRightLogicalNarrowingUpper

* ShiftRightLogicalRounded

* ShiftRightLogicalRoundedAdd

* ShiftRightLogicalRoundedAddScalar

* ShiftRightLogicalRoundedNarrowingLower

* ShiftRightLogicalRoundedNarrowingSaturateLower

* ShiftRightLogicalRoundedNarrowingSaturateUpper

* ShiftRightLogicalRoundedNarrowingUpper

* ShiftRightLogicalRoundedScalar

* ShiftRightLogicalScalar

* SignExtendWideningLower

* SignExtendWideningUpper

* ZeroExtendWideningLower

* ZeroExtendWideningUpper

* ShiftArithmeticRoundedSaturateScalar

* ShiftArithmeticSaturateScalar

* ShiftLeftLogicalSaturateScalar

* ShiftLeftLogicalSaturateUnsignedScalar

* ShiftLogicalRoundedSaturateScalar

* ShiftLogicalSaturateScalar

* ShiftRightArithmeticNarrowingSaturateScalar

* ShiftRightArithmeticNarrowingSaturateUnsignedScalar

* ShiftRightArithmeticRoundedNarrowingSaturateScalar

* ShiftRightArithmeticRoundedNarrowingSaturateUnsignedScalar

* ShiftRightLogicalNarrowingSaturateScalar

* ShiftRightLogicalRoundedNarrowingSaturateScalar

* BigMul with 64-bit operands (#35975)

* Unsigned 128-bit BigMul

* Signed 128-bit BigMul

* Added explanation of alg and reference to it

* Removed else branch

Co-authored-by: Adeel Mujahid <[email protected]>

* Optimize BitOperations uses in CoreLib (#35650)

* make Log2 branch-free

* convert LeadingZeroCount uses to Log2

* switch Bmi1.TrailingZeroCount calls to BitOperations

* [jit] Return the compiled method and an unbox trampoline from ldvirtftn when calling ldvirtftn on a valuetype method. (#36736)

Fixes https://github.com/dotnet/runtime/issues/34379.

<!--
Thank you for your Pull Request!

If you are new to contributing to Mono, please try to do your best at conforming to our coding guidelines http://www.mono-project.com/community/contributing/coding-guidelines/ but don't worry if you get something wrong. One of the project members will help you to get things landed.

Does your pull request fix any of the existing issues? Please use the following format: Fixes #issue-number
-->

Co-authored-by: vargaz <[email protected]>

* Revert "Libunwind1.5rc2 (#36027)" (#36909)

This reverts commit 8c6c7655bb7abc5c4ce769923aa5bea1daee6bd3.

* Remove PreserveDependency from SR class (#36358)

* Use TARGET_64BIT instead of set of TARGET_<arch> definitions for CLRConfig::EXTERNAL_CORECLR_PROFILER_PATH_64 and CLRConfig::EXTERNAL_CORECLR_PROFILER_PATH_32 checking. (#36924)

* Follow ups for Math.BigMul (#36920)

- Use more efficient and compact uint cast instead of bit masks where possible
- Improved test coverage
- Update comment in FastMod
- Optimize BigMul for 32-bit platforms

* Use BinaryPrimitives where possible. (#36881)

* Fix exception message to indicate Cofactor is required. (#36821)

* Disable GC/API/GC/AddThresholdTest on ARM64 (#36940)

Issue #36850

* Add comments per code review feedback

* Fixing emitAnyConst and related functions to better handle alignment (#36432)

* Fixing emitAnyConst and related functions to better handle alignment

* Don't use small alignment where it might not be supported

* Applying formatting patch

* Adding some comments explaining how emitAnyConst works

* Fix output for user strings in R2RDump (#36935)

Quote user strings and escape control characters, unpaired surrogates, and other unsafe characters.

* [master] Update dependencies from mono/linker Microsoft/vstest dotnet/xharness (#36811)

* Update dependencies from https://github.com/mono/linker build 20200520.1

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20269.1 -> To Version 5.0.0-preview.3.20270.1

* Update dependencies from https://github.com/microsoft/vstest build 20200521-01

Microsoft.NET.Test.Sdk
 From Version 16.7.0-preview-20200518-01 -> To Version 16.7.0-preview-20200521-01

* Update dependencies from https://github.com/dotnet/xharness build 20200520.2

Microsoft.DotNet.XHarness.Tests.Runners
 From Version 1.0.0-prerelease.20270.1 -> To Version 1.0.0-prerelease.20270.2

* Update dependencies from https://github.com/mono/linker build 20200521.1

Microsoft.NET.ILLink.Tasks
 From Version 5.0.0-preview.3.20269.1 -> To Version 5.0.0-preview.3.20271.1

* Update dependencies from https://github.com/dotnet/xharness build 20200521.3

Microsoft.DotNet.XHarness.Tests.Runners
 From Version 1.0.0-prerelease.20270.1 -> To Version 1.0.0-prerelease.20271.3

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Remove volatile on field in ConcurrentDictionary.Tables (#36976)

* configure.ac: check for sockaddr_in6 on Win32 (#36742)

HAVE_STRUCT_SOCKADDR_IN6 needs to be defined to enable IPv6 features in
mono/metadata/w32socket.c.

Fixes TcpListener issues in wine-mono.

Co-authored-by: w-flo <[email protected]>

* Update dependencies from https://github.com/dotnet/xharness build 20200525.1 (#36982)

Microsoft.DotNet.XHarness.Tests.Runners
 From Version 1.0.0-prerelease.20271.3 -> To Version 1.0.0-prerelease.20275.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Timeout support for unprivileged Ping on Linux/BSD  (#35461)

* Added support of timeout when unprivileged ping is used

* Fixed test according with new timeout parameter

* Fixes code style issues

* Fixed command-line flag for ping6 on BSD

* Separated timeout test

* Removed trailing whitespace

* Workaround for OSX ping6

* Removed static local method

* Suppress Ping stdout

* Fixed variable name

* Added test for zero timeout

* Handle zero timeout correctly

* Fix accumulating the offset in PemReader

The preebEndIndex offset was a total value in to the PEM's contents,
not a moving offset. This would cause it to advance past the contents
of the PE…
@ghost ghost locked as resolved and limited conversation to collaborators Dec 9, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants