Install clang-tools-9 on Ubuntu 16.04 and 18.04 cross-building images. #375

imhameed · 2020-10-22T18:54:12Z

clang-tools-9 depends on libclang-1-9 (among other things), which
contains libclang. Mono's offsets-tool generates data structure access
offsets for cross-compiled target architectures by parsing Mono headers
using libclang.

clang-tools-9 depends on libclang-1-9 (among other things), which contains libclang. Mono's offsets-tool generates data structure access offsets for cross-compiled target architectures by parsing Mono headers using libclang.

…talled. See also: dotnet/dotnet-buildtools-prereqs-docker#375

This change: - Adds new options to mono.proj: - MonoAOTEnableLLVM, which enables (or disables) building a Mono AOT cross compiler with LLVM; - MonoAOTLLVMDir, which specifies the path to a copy of LLVM suitable for building a Mono AOT cross compiler, and is optional; and - BuildMonoAOTCrossCompilerOnly, which allows building a Mono AOT cross compiler without building an associated full Mono runtime. - Changes offsets-tool.py's user interface slightly; '--include-prefix' is renamed to '--prefix' and may be specified multiple times. While this latter feature isn't necessary to build a Mono cross compiler today, because we don't use distribution-supplied cross-compilation headers on CI, it does make it easier to experiment with the offsets tool using arbitrary header layouts. For example, Debian's arm64 cross-compilation packages scatter useful header files across `/usr/lib/gcc-cross/aarch64-linux-gnu` and `/usr/aarch64-linux-gnu`. - Updates the docker image used for arm64 cross-compilation to a newer revision that includes libclang (see also dotnet/dotnet-buildtools-prereqs-docker#375). - Adds some tests that currently fail to compile with Mono LLVM AOT to issues.targets. - Adds a Linux_arm64 LLVM AOT job to CI. Nothing particularly fancy is done to build the Mono LLVM AOT cross compiler; it is built as a step in the same job that also sends Linux_arm64 tests to Helix.

This change: - Adds new options to mono.proj: - MonoAOTEnableLLVM, which enables (or disables) building a Mono AOT cross compiler with LLVM; - MonoAOTLLVMDir, which specifies the path to a copy of LLVM suitable for building a Mono AOT cross compiler, and is optional; and - BuildMonoAOTCrossCompilerOnly, which allows building a Mono AOT cross compiler without building an associated full Mono runtime. - Changes offsets-tool.py's user interface slightly; '--include-prefix' is renamed to '--prefix' and may be specified multiple times. While this latter feature isn't necessary to build a Mono cross compiler today, because we don't use distribution-supplied cross-compilation headers on CI, it does make it easier to experiment with the offsets tool using arbitrary header layouts. For example, Debian's arm64 cross-compilation packages scatter useful header files across `/usr/lib/gcc-cross/aarch64-linux-gnu` and `/usr/aarch64-linux-gnu`. - Updates the docker image used for arm64 cross-compilation to a newer revision that includes libclang (see also dotnet/dotnet-buildtools-prereqs-docker#375). - Adds some tests that currently fail to compile with Mono LLVM AOT to issues.targets. - Adds a Linux_arm64 LLVM AOT job to CI. Nothing particularly fancy is done to build the Mono LLVM AOT cross compiler; it is built as a step in the same job that also sends Linux_arm64 tests to Helix. Co-authored-by: imhameed <[email protected]>

* Arm32 Crossgen2 initial support (#43243) - Fix type layout bugs - Sequential or Explicit layout classes without explicit field offsets on arm32 should align their fields based on the start of the field list of the object - The field base offset used for R2R calculation on Arm32 should respect the RequiresAlign8 flag - Computing true for requiresAlign8 in auto field layout should set the alignment of a class to 8 during auto layout - if a class derives from an type which requires 8 byte alignment, set the derived to require higher alignment - Align the EH info table on 4 byte boundaries - Set the thumb bit on the arm32 personality routine RVA in XData - Enable Crossgen2 smoke test for arm - Adjust architecture specific type layout tests to match CoreCLR behavior - Fix alignment of Export functions within PE file * Remove unsafe code from System.Web.HttpUtility (#43422) * Fix the android cmake build. (#43421) * [browser][crypto] Remove restraining not supported attribute Primitives (#43387) * [browser][crypto] Remove restraining not supported attribute - The modules included within the System.Security.Cryptography.Primitives module should still be available for use outside of browser os. * Address review comments. Remove `<IncludePlatformAttributes>` attribute as well * Fix StaticTestGenerator (#43432) It's rotted a bit. * Clean up DependencyModel Json read/write (#43376) * Clean up DependencyContextWriter - Remove UnifiedJsonWriter - Remove ArrayBufferWriter and write to the Stream directly * Clean up DependencyContextJsonReader - Remove UnifiedJsonReader - Move any reader logic to extension methods * Add MetadataToken getter override to builder classes (#43330) * [mono] Include hostpolicy/hostfxr in mono desktop runtime packs (#42729) Currently mono desktop runtime packs don't include `libhostfxr.*` and `libhostpolicy*` libs needed for corehost in self contained mode. Co-authored-by: Alexander Köplinger <[email protected]> * Remove some unsafe code from Console (#43368) * Remove some unsafe code from Console * preserve original byte-by-byte decoding * remove unsafe declaration where not needed Co-authored-by: Stephen Toub <[email protected]> * Bind byte array from base64 config value (#43150) * Bind byte array from base64 config value * Add test case for failure * Add test for null case * Remove unnecessary null check Co-authored-by: Thomas Levesque <[email protected]> * Add w32subset.h into monoutils_sources. (#43435) Co-authored-by: lateralusX <[email protected]> * [runtime] Add an options API. (#32595) * [runtime] Add an options API. Add a general options API to the runtime, based on the flags API in Google V8: ```https://chromium.googlesource.com/v8/v8.git/+/refs/heads/master/src/flags/``` Supported features: * Definition of runtime options in a declarative way. * Options are mapped to C globals. * BOOL/INT/STRING data types. * Generic option parsing code. * Generic usage code. * Read-only flags for build-time optimization. This is designed to replace the many option parsing functions in the runtime, MONO_DEBUG, the many mono_set_... functions etc. * Fix the build. Co-authored-by: vargaz <[email protected]> Co-authored-by: Zoltan Varga <[email protected]> * Revert "add better handling of SECBUFFER_EXTRA during TLS handshake on Windows (#42427)" (#43442) This reverts commit 51f6b8bd3a2a38c432b1cd1f7c465c256f5f699c. * Fix runincontext testing (#43446) After some recent shuffles of tests location, running the coreclr tests with runincontext option stopped working. This change fixes it by fixing the script path in the run.py. * Delete CoreDllMain, remove DLL_THREAD_DETACH from EEDllMain, just rely on thread local destructor (#43423) * [debugger][wasm] Implement Debugger.IsAttached on wasm (#42532) * Debugger.IsAttached is now working on wasm. And can be used to detect if debugger is attached. Fix #42411 * Update src/mono/wasm/runtime/library_mono.js Co-authored-by: Ryan Lucia <[email protected]> * Using the infrastructure to not send dynamically loaded assemblies if debugger is not attached. Changing where to check if the assembly is already added to avoid unnecessary checks. * Checking the assembly name size. Co-authored-by: Ryan Lucia <[email protected]> * Add a makefile sample to run test-browser (#43382) * Add back nightly build table (#43392) * Update README table generator. * Link README to new table location. * Add 6.0 coreclr runtime links for table generation. * Add generated table to the dogfooding page. * Update some stale references to sleet feeds, older versions of the runtime, and deprecated packages. * Add subset for table generation and reorder table to frontload OS groups. Co-authored-by: Jan Kotas <[email protected]> * Delete superfluous suffix * JIT: ensure bbflags get treated as 64 bit literals (#43451) Expressions like `~(BBF_KEEP_BBJ_ALWAYS)` were being evaluated as 32 bit signed quantities, leading to mask value `00000000_7FFFFFFFF` instead of the desired `FFFFFFFF_7FFFFFFFF`, causing inadvertent clearing of flags with higher value. SPMI diffs showed the only flag loss that impacted codegen was `BBF_HAS_CALL`, which feeds into the CSE heuristics. So no known correctness issue, but it is certainly possible to also lose `BBF_DOMINATED_BY_EXCEPTIONAL_ENTRY` or `BBF_HAS_SUPPRESSGC_CALL` and that may be more serious. * Fix two tests for runincontext testing (#43457) * Disable two tests for runincontext testing These tests are not compatible with running using the runincontext tool. * Reflect PR feedback - make the test work instead * Remove unused Unix PKCS12 shims Because the PKCS#12/PFX import is now done with managed code, the PKCS12 shim was dead code. * Modify System.Net.HttpListener to throw PNSE at assembly level on browser (#43401) * JIT: some small profile related fixes (#43408) 1. If we're inheriting a fraction of the profile weight of a profiled block, mark the inheriting block as profiled. This prevents methods like `optSetBlockWeights` or `optMarkLoopBlocks` from coming along later and setting the weights to something else. Since the full inheritance method has similar logic, make it delegate to the fractional one, with a scale of 100 (no scaling). 2. If we switch from Tier0 to FullOpt, make sure to clear the BBINSTR flag, else we'll put probes into optimized code. 3. Dump edge weights in the dot graph, if we have them. 4. Only dump the flow graph twice per phase. * adjust renegotiation tests to match product change (#43123) * adjust renegotiation tests to match product change * add assert for validationCount * Add issues.targets exclusions for Crossgen2 Pri 1 Tests (#43473) This baselines the remaining test failures with GH issues to track for further investigation. * Fix new warnings (#43097) * Fix CA1416 warnings in runtime repo * Port ACL OpenExisting overloads for EventWaitHandle/Mutex/Semaphore (#43134) * Add methods to ref file. * Add empty methods to src files. * Add the .NET Framework version of these methods for .NET Standard. * Move OpenExistingResult enum to Common and consume it where needed. * Remove Unix comment in Windows-only file. * Document OpenExistingResult enum. * Make out result parameters nullable. * Add exception resource string. * Imlement EventWaitHandleAcl methods. * Implement MutexAcl methods. * Implement SemaphoreAcl methods. * Remove unnecessary check for null or empty name. * Document the EventWaitHandleAcl methods. * Document the MutexAcl methods. * Document the SemaphoreAcl methods. * Add negative enum check. Fix incorrect cast. * Add NotNullWhen attribute to TryOpenExisting out parameter. Adjust docs. * Add EventWaitHandle basic unit tests. * Add Semaphore basic unit tests * Add Mutex basic unit tests. * Add Mutex exception handling unit tests. * Add EventWaitHandle exception handling unit tests. * Add Semaphore exception handling unit tests. * EventWaitHandle and Mutex throw DirectoryNotFoundException when PathNotFound. Adjust documentation. * Nullability in ref file. * Do not check for rights out of range value, let Windows handle it. Adjust unit tests accordingly. * Spacing. * Remove enum range test. Add PathNotFound tests. Co-authored-by: Carlos Sanchez Lopez <[email protected]> * Fix xunit analyzers to run on library test projects (#43459) We don't want most analyzers running over our test code currently (some rules could be enabled with varying degrees of effort), but we do want the xunit analyzers running, and they haven't been. Fix that by creating a new ruleset specific to library tests, and switching over to use it when building library test projects. * Remove some unnecessary unsafe usage (#43430) * Fix nullable warnings in struct constructors (#43472) * Add additional URI schemes (WIP) (#43375) * Remove unused Common Extensions code (#43452) * Make more suitable SPC instance methods static (#43280) Co-authored-by: Jan Kotas <[email protected]> * Enable Mono substitutions and attributes exclusion for mobile (#43507) * [mono] Ensure MonoAssemblyName is in sync between managed and native (#43536) We also no longer appear to need the NETCORE or DISABLE_REMOTING defines in msbuild, so remove them * Port changes from dotnet/runtimelab (#43496) - Fix build errors in System.Globalization.Native with libraries/Native warning level, add System.Globalization.Native back to the libraries/Native build to protect it, - Misc other changes * test for 2164 and corert 8246 (#43511) * Ongoing cmake build work. (#43519) * Ongoing cmake build work. * Fix llvm support when cross compiling * Fix/enable ios support. * Add support for amd64->arm/arm64 cross builds on CI. * [master] Update dependencies from dotnet/arcade dotnet/xharness dotnet/llvm-project dotnet/icu mono/linker (#43355) Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Viktor Hofer <[email protected]> Co-authored-by: Jeremy Koritzinsky <[email protected]> Co-authored-by: Marek Safar <[email protected]> * [threads] At shutdown, don't wait for native threads that aren't calling Mono (#43174) * [test] Call P/Invoke callback delegates from foreign threads Change post-detach-1.cs to also have versions that call reverse pinvokes from foreign threads that were not attached to mono. The foreign threads should not prevent GC and should not prevent Mono from shutting down using mono_manage_internal. * [threads] Don't wait for native threads that aren't calling Mono If a thread is started from native code, and it interacts with the runtime (by calling a thunk that invokes a managed method), the runtime will attach the thread - it will create a `MonoInternalThread` and add it to the list of threads hash (in threads.c). If the thread returns from the managed call, it will still be recorded by the runtime, but as long as it is not running managed code anymore, it will prevent shutdown. The problem is when we try to suspend the thread in order to abort it, `mono_thread_state_init_from_handle` will see a NULL domain (because `mono_threads_detach_coop_internal` will restore it to NULL when a managed method returns back to native code). (On systems using POSIX signals to suspend, the same check is in `mono_thread_state_init_from_sigctx`). As a result, `mono_threads_suspend_begin_async_suspend` (or `suspend_signal_handler` on POSIX) will set `suspend_can_continue` to FALSE, and `mono_thread_info_safe_suspend_and_run` will not run the suspend callback. As a result, when `mono_manage_internal` calls `abort_threads`, it will add the thread handle to the wait list, but it will not actually request the thread to abort. As a result, after `abort_threads` returns, the subsequent call to `wait_for_tids` will block until the native thread terminates (at which point the TLS destructor will notify the thread handle and wait_for_tids will unblock). This commit changes the behavior of `abort_threads` to ignore threads that do not run `async_suspend_critical` and not to add them to the wait list. As a result, if a native thread calls into managed and then returns to native code, the runtime will not wait for it. * [threads] Warn if mono_thread_manage_internal can't abort a thread Give a hint to embedders to aid debugging * rename AbortThreadData:thread_will_abort field It's used to keep track of whether the thread will eventually throw a TAE (and thus that we need to wait for it). The issue is that under full coop suspend, we treat threads in GC Safe (BLOCKING) state as if they're suspended and always execute async_abort_critical. So the field has nothing to do with whether the thread was suscessfully suspended, but rather whether it will (eventually) be aborted. * [threads] Fix async_abort_critical for full coop If the foreign external thread doesn't have any managed methods on its callstack, but it once called a native-to-managed wrapper, it will be left by mono_threads_detach_coop in GC Safe (BLOCKING) state. But under full coop, GC Safe state is considered suspended, so mono_thread_info_safe_suspend_and_run will run async_abort_critical for the thread. But the thread may never call into Mono again, in which case it will never safepoint and aknowledge the abort interruption. So set thread_will_abort to FALSE in this case, so that mono_thread_manage_internal won't try to wait for it. --- Related to an issue first identified in https://github.com/mono/mono/pull/18517 --- This supersedes mono/mono#18656 Co-authored-by: lambdageek <[email protected]> * add better handling of SECBUFFER_EXTRA during TLS handshake on Windows (#43475) * add better handling of SECBUFFER_EXTRA during TLS handshake on Windows (#42427) * add better handling of SECBUFFER_EXTRA during TLS handshake on Windows * fix boundery check * fix spelling * update Authentication_IncorrectServerName_Fail test * feedback from review * Update src/libraries/System.Net.Security/tests/FunctionalTests/CertificateValidationRemoteServer.cs Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Stephen Toub <[email protected]> * Convert more interop to use function pointers (#43514) * Fix TimeZoneInfo to handle Yukon zone (#43550) * Fix async iterators to clear out state upon completion (#43522) AsyncIteratorMethodBuilder was only doing its clean-up for completion (e.g. zeroing out the state machine and context, removing the object from a debugger-incited tracking table) if the last call to the iterator was part of asynchronous completion; if the last MoveNextAsync completed synchronously, the used code path could miss that cleanup work. * Add test to validate precedence of roll-forward and roll-forward-on-no-candidate-fx settings (#43510) The comment on the test documents the desired precedence order. Moved some of the existing tests which validate the behavior of the two settings together into a new separate test class. * Refactor NonAscii bit mask usage (#43537) * Refactor NonAscii bit mask usage The Vector128 type is being left in a Blazor WASM application because Utf8Utility.GetPointerToFirstInvalidByte is always creating one, even though it isn't used. I refactored the code such that the bit mask is no longer created on platforms it is not used, since it is only being used by Arm64. * Improve performance of polymorphism (#42538) * Use Span-based CreateHMAC where possible * Add crossBuild parameter to yaml (#43319) * Add crossBuild parameter to yaml * Use NetCorePublic-Pool pool instead of AzDO hosted pool for Browser jobs (#43589) The hosted pool runs into no disk space issues. * Equals and GetHashCode for Reflection.Pointer (#42547) * JIT: don't inline methods with small stackallocs if the call site is … (#43516) The logic in `fgInlinePrependStatements` that zero-initializes locals doesn't kick in for jit temps introduced when small stackallocs are optimized. So if we inline a method with a small stackalloc into a loop, the memory for the stackalloc doesn't get properly re-zeroed on each iteration. Fix by disallowing such inlines by adding an extra check: the call site must not be in a loop. Closes #43391. * Fix CG2 outerloop comparison runs and OSX leg warning (#43547) The outerloop run has OSX checked test runs for CG2 and CG2 composite modes. Currently both would use a log upload artifact with the same name (coreclr__TestRunLogs_R2R_CG2_OSX_x64_checked_outerloop). Disambiguate the two with a different `LogNamePrefix` for composite runs. The crossgen2 comparison runs are failing to create the baseline crossgen'd framework because the live libraries zip's internal path has changed. .net5 has been replaced with .net6. The build should have failed when we tried to copy from the wrong folder but the error got eaten and we ended up with a malformed framework folder. Adjust the inline yml scripts so if they fail, it will fail that containing build task. * First draft of Dynamic Pgo proposal (#43371) First draft of Dynamic Pgo proposal. * Remove UnsupportedOsPlatform from CryptoConfig as a utility class (#43611) * Remove UnsupportedOsPlatform from CryptoConfig as it is a safe utility class * Clean up the other references * Add CSV map file generation for compiler diagnostics (#43612) Add `--csvmap` switch to Crossgen2 which causes it to generate node summary CSV files that are parsable by tests. The intent is to use this for size on disk perf tests so just the node type statistics and individual node map are implemented in CSV files. We can add section and relocs easily if we think they'll be useful in future. * add activity support for android sample (#43504) * [debugger] Switch to GC Unsafe in signal handler callbacks (#43600) If the runtime gets a single step or breakpoint signal while it is already running native code for a P/Invoke, it will be in GC Safe mode. Switch back to GC Unsafe to run the debugger engine steps. Addresses https://github.com/mono/mono/issues/20490 Co-authored-by: lambdageek <[email protected]> * disable DefaultConnect_EndToEnd_Ok on Windows7 (#43628) * disable DefaultConnect_EndToEnd_Ok on Windows7 * update platform check * update platform check * [iOS] Add mono runtime and AppleAppBuilder pkgproj for iOS sample (#43048) * Add iOS sample pkgproj * Add iOS Sample pkgproj to descriptions * Add iOS sample pkgproj ProjectReference * Add project reference to build AppleAppBuiler.csproj * Fixup AppleAppBuilder assembly path * Add AppleAppBuilder Packaging target * Remove UI file to allow and encourage sample users to modify the UI * Move package from dotnet6-transport to dotnet6 feed Co-authored-by: Mitchell Hwang <[email protected]> * [System.IO.Compression] ZipHelper.DosTimeToDateTime handle empty LastModified fields in zip archive entry header without internal exception (#43008) * [System.IO.Compression] ZipHelper.DosTimeToDateTime handle empty LastModified field without internal exception to improve debugging performance on several zip files opening asynchronously * [Test][Compression][ZipArhiveEntry] Add unit test to test empty lastModified field in zip entry * refactor unit test code * do not use arraypool in tests * fix test after azure pipeline checks with errors * fix invalid assert in new test * improve usability of NegotiateStreamInvalidOperationTest (#43622) * [browser][debugger] Clean up MessageId logic to prepare for sessions in the test harness (#43188) * Clean up MessageId logic * Update src/mono/wasm/debugger/DebuggerTestSuite/InspectorClient.cs Co-authored-by: Ankit Jain <[email protected]> * Update src/mono/wasm/debugger/DebuggerTestSuite/InspectorClient.cs Co-authored-by: Ankit Jain <[email protected]> * Update src/mono/wasm/debugger/DebuggerTestSuite/InspectorClient.cs Co-authored-by: Ankit Jain <[email protected]> Co-authored-by: Ankit Jain <[email protected]> * Fixing stale version badges in docs (#43558) Fixing stale version badges in docs From the issue description: Improper cache-control in generated badges caused browsers caching svg badges for a year. It is, therefore, recommend to modify related README.md content by simple find & replace from: _version_badge.svg to _version_badge.svg?no-cache This will trigger github to compute and use different/new Camo proxy URL. Fix #3822 * improve reliability of SslStream tests with failing certificate validation (#43570) * improve reliability of SslStream tests with failing certificate validation * Update src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs Co-authored-by: Stephen Toub <[email protected]> * fix failing ALPN test on old OpenSSL Co-authored-by: Stephen Toub <[email protected]> * Mono: Turn on cmake build by default (#42119) Turns it on everywhere except Windows. * Sync shared code from aspnetcore (#43624) * Use ThrowHelper inside ValueType types (#43634) * Display the real offset after branch tightening (#43469) * display the real offset after branch tightening * emit offset for cold codeblock * move isColdBlock inside #DEBUG * fix formatting * Revert "fix formatting" This reverts commit 0744e1e432e12c5bddd4ce8c208660e31d96f12a. * Revert "move isColdBlock inside #DEBUG" This reverts commit 60e511b0bfb1836476a5fcf05ff3450d32c3d907. * Revert "emit offset for cold codeblock" This reverts commit e760beb2fcd1a5fcfd22ddf47eb6939a7094562d. * Use emitCurCodeOffs() to display correct offset * Clarify pre-merge commit squash guidance (#43664) * Update MulticoreJit (#39996) - Enable Generic Methods in MulticoreJit - Enable NDirect Stub in MulticoreJit * Fix missing signatures for Cross bitness DAC symbols (#43500) * Unify paths used for cross-bit components * Pass down buildArchitecture for signing * Pass target properties to signing * Automatically trigger the ilasm round-trip test on PRs (#43666) For changes to the ilasm or ildasm source code, trigger the ilasm round-trip pipeline. There could, of course, be other changes that could affect ilasm/ildasm, but this at least catches the primary ones. * Mono: Fix the windows cmake build. (#43658) This will not use cmake on windows, it just fixes the conditionals so the windows build doesn't fail if cmake is enabled by default. * [Browser] don't pass redundant args to wasm (#41608) * Enable implicit fallthrough warning (#43397) * Enable implicit fallthrough warning This change enables warnings for implicit fallthrough in switch cases and fixes all the cases where the warning was reported. It also fixes some places where the fall through was incorrect. Fortunately, these places were not causing functional issues. * Disable / fix failing test in runincontext testing (#43663) * Disable all profiler tests since they launch a secondary process and process launch creates an infinite event loop in the SocketAsyncEngine on Linux. Since runincontext loads even framework assemblies into the unloadable context, locals in this loop prevent unloading. The tests were working before Process.Start moved to using sockets. * Fix the multifoldertest to work under runincontext - the shell script generated from the .csproj was passing an absolute path for the multifolder.dll to the runincontext.sh/cmd instead of a relative path that is used in all other tests and that the runincontext expects. * Add test leg to the PR build to run libraries tests on Android emulators (#37585) * Add public JsonElement.ParseValue() and TryParseValue() (#43601) * [mono] Copy image data with AssemblyLoadContext.LoadFromStream (#43592) We don't actually pin the byte array, so it must be copied or it can be overwritten once we run a GC on the LOH. Fixes https://github.com/dotnet/runtime/issues/43402 Tested manually that it fixes the issue using the associated repro. This isn't really something that lends itself to a test, so that's the best I can do. * Fix incremental build of tasks.proj for mobile (#43674) * Fix incremental build of tasks.proj for mobile The semaphore file that is used as the input for deciding whether to rebuild tasks.proj wasn't properly taking the mobile task projects into account. This resulted in e.g. WasmAppBuilder not being built if you built for desktop before, resulting in a build error. We now use the conditioned project references as an input instead of globbing through all nested projects. Co-authored-by: Viktor Hofer <[email protected]> * [master] Update dependencies from mono/linker dotnet/arcade dotnet/xharness dotnet/llvm-project (#43583) * Update dependencies from https://github.com/dotnet/arcade build 20201015.7 Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.ApiCompat From Version 6.0.0-beta.20514.1 -> To Version 6.0.0-beta.20515.7 * Update dependencies from https://github.com/dotnet/xharness build 20201019.2 Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 1.0.0-prerelease.20516.1 -> To Version 1.0.0-prerelease.20519.2 * Update dependencies from https://github.com/mono/linker build 20201020.1 Microsoft.NET.ILLink.Tasks From Version 6.0.0-alpha.1.20516.1 -> To Version 6.0.0-alpha.1.20520.1 * Update dependencies from https://github.com/mono/linker build 20201020.2 Microsoft.NET.ILLink.Tasks From Version 6.0.0-alpha.1.20516.1 -> To Version 6.0.0-alpha.1.20520.2 * Update dependencies from https://github.com/dotnet/arcade build 20201016.5 Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.ApiCompat From Version 6.0.0-beta.20514.1 -> To Version 6.0.0-beta.20516.5 * Update dependencies from https://github.com/dotnet/llvm-project build 20201019.1 runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk From Version 9.0.1-alpha.1.20512.1 -> To Version 9.0.1-alpha.1.20519.1 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * RyuJIT: Don't emit null checks for constant strings (#37245) * Fold "ldstr ==/!== null" (mikedn's PR) * Formatting * check IsUnsigned for GT_GT * Address feedback * Update steps in debugging instruction of CLR (#43557) * Update CLR debugging instruction steps. * Use suggested BCL path Co-authored-by: Jan Vorlicek <[email protected]> * Mention the usage cases of CORE_LIBRARIES. Co-authored-by: Jan Vorlicek <[email protected]> * Remove non-cmake build support from mono.proj. (#43678) * [Android] Add AndroidAppBuilder pkgproj for Android sample (#43216) In preparation to bring mono samples to `dotnet/samples`, files that are most likely to change will be packaged into a nuget package to be downloaded and consumed on the `dotnet/samples` end rather than mirroring changes. This PR expands the process that created the NuGet package for `Microsoft.NETCore.BrowserDebugHost.Transport` to build a NuGet package for the Android sample. Co-authored-by: Mitchell Hwang <[email protected]> * Add test leg to the PR build to run libraries tests on Android devices (#42209) The on-device version of https://github.com/dotnet/runtime/pull/37585 Co-authored-by: Premek Vysoky <[email protected]> Co-authored-by: Santiago Fernandez Madero <[email protected]> * Migrate coreclr's worker thread pool to be able to use the portable thread pool in opt-in fashion (#38225) - Enables using the portable thread pool with coreclr as opt-in. Change is off by default for now, and can be enabled with COMPlus_ThreadPool_UsePortableThreadPool=1. Once it's had bake time and seen to be stable, at a reasonable time in the future the config flag would ideally be removed and the relevant parts of native implementation deleted. - The IO thread pool is not being migrated in this change, and remains on the native side - My goal was to get compatible behavior, compatible with diagnostics tools, and similar perf to the native implementation in coreclr. Tried to avoid changing scheduling behavior, behavior of heuristics, etc., compared with that implementation. - The eventual goal is to have one mostly managed thread pool implementation that can be shared between runtimes, to ease maintenance going forward Commit descriptions: - "Add dependencies" - Ported LowLevelLock from CoreRT, and moved LowLevelSpinWaiter to shared. Since we support Thread.Interrupt(), they were necessary in the wait subsystem in CoreRT partly to support that, and were also used in the portable thread pool implementation where a pending thread interrupt on a thread pool thread would otherwise crash the process. Interruptible waits are already used in the managed side of the thread pool in the queue implementations. It may be reasonable to ignore the thread interrupt problem and suggest that it not be used on thread pool threads, but for now I just brought in the dependencies to keep behavior consistent with the native implementation. - "Add config var" - Added config var COMPlus_ThreadPool_UsePortableThreadPool (disabled by default for now) - Flowed the new config var to the managed side and set up a mechanism to flow all of the thread pool config vars - Removed debug-only config var COMPlus_ThreadpoolTickCountAdjustment, which didn't seem to be too useful - Specialized native and managed thread pool paths based on the config var. Added assertions to paths that should not be reached depending on the config var. - "Move portable RegisteredWaitHandle implementation to shared ThreadPool.cs" - Just moved the portable implementation, no functional changes. In preparation for merging the two implementations. - "Merge RegisteredWaitHandle implementations" - Merged implementations of RegisteredWaitHandle using the portable version as the primary and specializing small parts of it for coreclr - Fixed PortableThreadPool's registered waits to track SafeWaitHandles instead of WaitHandles similarly to the native implementation. The SafeWaitHandle in a WaitHandle can be modified, so it is retrieved once and reused thereafter. Also added/removed refs for the SafeWaitHandles that are registered. - "Separate portable-only portion of RegisteredWaitHandle" - Separated RegisteredWaitHandle.UnregisterPortable into a different file, no functional changes. Those paths reference PortableThreadPool, which is conditionally included unlike ThreadPool.cs. Just for consistency such that the new file can be conditionally included similarly to PortableThreadPool. - "Fix timers, tiered compilation, introduced time-sensitive work item queue to simulate coreclr behavior" - Wired work items queued from the native side (appdomain timer callback, tiered compilation background work callback) to queue them into the managed side - The timer thread calls into managed code to queue the callback - Some tiered compilation work item queuing paths cannot call managed code, so used a timer with zero due time instead - Added a queue of "time-sensitive" work items to the managed side to mimic how work items queued from the native side ran previously. In particular, if the global queue is backed up, when using the native thread pool the native work items still run ahead of them periodically (based on the Dispatch quantum). Potentially they could be queued into the global queue but if it's backed up it can potentially significantly and perhaps artificially delay the appdomain timer callback and the tiering background jitting. I didn't want to change the behavior in an observable (and potentially bad) way here for now, a good time to revisit this would be when IO completion handling is added to the portable thread pool, then the native work items could be handled somewhat similarly. - "Implement ResetThreadPoolThread, set thread names for diagnostics" - Aside from that, setting the thread names (at OS level) allows debuggers to identify the threads better as before. For threads that may run user code, the thread Name property is kept as null as before, such that it may be set without exception. - "Cache-line-separate PortableThreadPool._numRequestedWorkers similarly to coreclr" - Was missed before, separated it for consistency - "Post wait completions to the IO completion port on Windows for coreclr, similarly to before" - On Windows, wait completions are queued to the IO thread pool, which is still implemented on the native side. On Unixes, they are queued to the global queue. - "Reroute managed gate thread into unmanaged side to perform gate activites, don't use unmanaged gate thread" - When the config var is enabled, removed the gate thread from the native side. Instead, the gate thread on the managed side calls into the native side to perform gate activities for the IO completion thread pool, and returns a value to indicate whether the gate thread is still necessary. - Also added a native-to-managed entry point to request the gate thread to run for the IO completion thread pool - "Flow config values from CoreCLR to the portable thread pool for compat" - Flowed the rest of the thread pool config vars to the managed side, such that COMPlus variables continue to work with the portable thread pool - Config var values are stored in AppContext, made the names consistent for supported and unsupported values - "Port - ..." * 3 - Ported a few fixes that did not make it into the portable thread pool implementation - "Fix ETW events" - Fixed the EventSource used by the portable thread pool, added missing events - For now, the event source uses the same name and GUID as the native side. It seems to work for now for ETW, we may switch to a separate provider (along with updating tools) before enabling the portable thread pool by default. - For enqueue/dequeue events, changed to use the object's hash code as the work item identifier instead of the pointer since the pointer may change between enqueue and dequeue - "Fix perf of counts structs" - Structs used for multiple counts with interlocked operations were implemented with explicit struct layout and field offsets. The JIT seems to generate stack-based code for such structs and it was showing up as higher overhead in perf profiles compared to the equivalent native implementation. Slower code in compare-exchange loops can cause a larger gap of time between the read and the compare-exchange, which can also cause higher contention. - Changed the structs to use manual bit manipulation instead, and microoptimized some paths. The code is still not as good as that generated by C++, but it seems to perform similarly based on perf profiles. - Code size also improved in many cases, for example one of the larger differences was in MaybeAddWorkingWorker(), which decreased from 585 bytes to 382 bytes and with far fewer stack memory operations - "Fix perf of dispatch loop" - Just some minor tweaks as I was looking at perf profiles and code of Dispatch() - "Fix perf of ThreadInt64PersistentCounter" - The implementation used to count completed work items was using `ThreadLocal<T>`, which turned out to be too slow for that purpose according to perf profiles - Changed it to rely on the user of the component to provide an object that tracks the count, which the user of the component would obtain from a ThreadStatic field - Also removed the thread-local lookup per iteration in one of the hot paths in Dispatch() and improved inlining - "Miscellaneous perf fixes" - A few more small tweaks as I was looking at perf profiles and code - In ConcurrentQueue, added check for empty into the fast path - For the portable thread pool, updated to trigger the worker thread Wait event after the short spin-wait completes and before actually waiting, the event is otherwise too verbose when profiling and changes performance characteristics - Cache-line-separated the gate thread running state as is done in the native implementation - Accessing PortableThreadPool.ThreadPoolInstance multiple times was generating less than ideal code that was noticeable in perf profiles. Tried to avoid it especially in hot paths, and in some cases where unnecessary for consistency if nothing else. - Removed an extra call to Environment.TickCount in Dispatch() per iteration - Noticed that a field that was intended to be cache-line-separated was not actually being separated, see https://github.com/dotnet/runtime/issues/38215, fixed - "Fix starvation heuristic" - Described in comment - "Implement worker tracking" - Implemented the equivalent in the portable thread pool along with raising the relevant event - "Use smaller stack size for threads that don't run user code" - Using the same stack size as in the native side for those threads - "Note some SOS dependencies, small fixes in hill climbing to make equivalent to coreclr" - Corresponds with PR that updates SOS: https://github.com/dotnet/diagnostics/pull/1274 - Also fixed a couple of things to work similarly to the native implementation - "Port some tests from CoreRT" - Also improved some of the tests - "Fail-fast in thread pool native entry points specific to thread pool implementations based on config" - Scanned all of the managed-to-native entry points from the thread pool and thread-entry functions, and promoted some assertions to be verified in all builds with fail-fast. May help to know in release builds when a path that should not be taken is taken and to avoid running further along that path. - "Fix SetMinThreads() and SetMaxThreads() to return true only when both changes are successful with synchronization" - These are a bit awkward when the portable thread pool is enabled, because they should return true only when both changes are valid and return false without making any changes otherwise, and since the worker thread pool is on the managed side and IO thread pool is on the native side - Added some managed-to-native entry points to allow checking validity before making the changes, all under a lock taken by the managed side - "Fix registered wait removals for fairness since there can be duplicate system wait objects in the wait array" - Described in comment - "Allow multiple DotNETRuntime event providers/sources in EventPipe" - Temporary change to EventPipe to be able to get events from dotnet-trace - For now, the event source uses the same name and GUID as the native side. It seems to work for now for ETW, and with this change it seems to work with EventPipe for getting events. Subscribing to the NativeRuntimeEventSource does not get thread pool events yet, that is left for later. We may switch to a separate provider (along with updating tools) before enabling the portable thread pool by default, as a long-term solution. - "Fix registered wait handle timeout logic in the wait thread" - The timeout logic was comparing against how long the last wait took and was not timing out waits sometimes, fixed to consider the total time since the last reset of timeout instead - "Fix Browser build" - Updated the Browser-specific thread pool variant based on the other changes Corresponding PR to update SOS: https://github.com/dotnet/diagnostics/pull/1274 Fixes https://github.com/dotnet/runtime/issues/32020 * Delete NetEventSource.Fail (#43579) At some point some Debug.Asserts/Fails were replaced by this NetEventSource.Fail helper, which both Debug.Fails and fires an EventSource event. But asserts in our code base are intended for things that should never happen, and we needn't be emitting events for them (if we did want to emit events for them, we'd need to tackle the other ~20,000 Debug.Assert/Fails in the codebase. I've deleted NetEventSource.Fail, and fixed up the call sites. Some were simply replaced by Debug.Assert/Fail. Some were deleted entirely, when from code inspection it looked like they could actually be hit, but were guarded by a check for the event source being enabled and thus were unlikely to have been triggered in our previous testing. Etc. * Code sample for supporting dynamic objects (#42097) * Fix crossgen2 armel build (#42811) Build fails with libjitinterface_armel.so and libclrjit_unix_armel_x64.so was not found after #41126. Signed-off-by: Timur <[email protected]> * Fix fallthrough cases in portable thread pool change (#43701) * Fix and test crossgen2 on arm and x86 (#42998) Fix last issues preventing the Pri0 tests from passing under crossgen2 for arm and x86 - Add support for stackprobe helper on arm32 - Fix field layout for x86 structures that contain long enums - Add test suite for these enum scenarios - Report the same alignment info to the jit for crossgen2 based compilation as was done in the core runtime Also enable testing targetting x86 and arm * [llvm] Fix some simd issues. (#43647) * [llvm] Add support for MONO_TYPE_FTNPTR. * [llvm] Fix support for some SIMD intrinsics. * Always use OP_SSE41_ROUNDS with 2 arguments, JIT opcodes can't have optional arguments. * Convert arguments to intrinsic calls, sometimes they have slighly different pointer or vector types. * [jit] Refactor the SIMD intrinsics handling code to share more code. Avoid emitting LLVM intrinsics which are not not enabled since it would cause llc to fail. * Replace command-line parser for Crossgen2 (#43655) Switch back to the old command-line parser for Crossgen2 to improve performance of parsing arguments. * Extend allowed Task.Delay/CTS TimeSpan values to match Timer (#43708) For some reason, Task.Delay(TimeSpan, ...) and CancellationTokenSource.CancelAfter(TimeSpan) cut off the max allowed timeout at int.MaxValue milliseconds, whereas Timer's TimeSpan support (which is used under the covers) goes all the way to UInt32.MaxValue - 2. This changes Task/CancellationTokenSource to match, effectively doubling the allowed length of the timeouts. * Slim down Path::Combine by removing dependency on ValueTuple`8 (#43582) Co-authored-by: Jan Kotas <[email protected]> * Fix issue #43714. Guid fallback for BigEndian had icorrect offsets. (#43718) * Align MailAddress.GetHashCode with Equals (#43573) * Bring back missing Exit_Failure case (#43665) This script label and code was removed in a previous change even though it is still required. Bring it back. * Fix ildasm of certain floating-point numbers (#43673) Change #42848 altered comments to print numbers in comments in little-endian format, but went too far and changed two places that print out numbers outside of comments that are later parsed by ilasm in round-trip testing. Fixes #43672 * Fix perf issue in System.Diagnostics.Activity (#43710) * Update "Input Image Architectures" table in r2rdump README.md (#43727) * gc: shorten background thread name to fit Linux name limit (#43679) * gc: shorten background thread name to fit Linux name limit * Allow dynamic code sample to compile on v3.0-5.0 (#43703) * [jit] Fix gsharedvt constrained calls to Object.GetType () under netcore. (#43729) * [jit] Fix gsharedvt constrained calls to Object.GetType () under netcore. Fixes https://github.com/dotnet/runtime/issues/35674. * Update src/mono/mono/mini/jit-icalls.c Co-authored-by: Aleksey Kliger (λgeek) <[email protected]> Co-authored-by: Aleksey Kliger (λgeek) <[email protected]> * Change list-processed to ps (#42293) (#42297) Co-authored-by: Sunguoyun <[email protected]> * Increase Hosting tests timeout (#43695) * Increase Hosting tests timeout There are some environments (checked CoreCLR, no tiered compilation) where the current timeout is insufficient. Increasing the timeout so the tests don't fail in these environments. Fix #43389 * Make DebugDirectoryBuilder.AddCodeViewEntry public (#43267) * Trim quoted file names passed to Crossgen2 (#43746) * Fixes to allow using OS-provided threadpool (#43726) Backport from dotnet/runtimelab:NativeAOT * Move Thread PNSE check to managed to cut more dependencies (#43730) * Disable test against https://github.com/dotnet/runtime/issues/43754 (#43755) * Fix the signature of Interop.Sys.Log () to match the native signature. (#43744) Signature mismatches cause errors on some platforms like wasm. * Add missing opcodes to k_rgnStackPushes (#42246) The rewriter defined two extra opcodes, CEE_COUNT and CEE_SWITCH_ARG, but does not define them in the k_rgnStackPushes array. This can cause out-of-bounds reads when computing the value of maxstack. * [mono] Use CMAKE_CURRENT_SOURCE_DIR instead of CMAKE_SOURCE_DIR (#43715) CLion creates a directory named ".idea" as a sibling to the top-level CMakeLists.txt that describes the project; with src/mono, git clean -dXf will delete src/mono/.idea because ".idea/" is an ignored pattern in .gitignore. One workaround is to create an out-of-tree CMakeLists.txt that contains nothing but add_directory(relative/path/to/src/mono), but this changes the value of CMAKE_SOURCE_DIR to something other than what our CMake build files expect. * Allow WasmAppBuilder to run after the publish step. Change the (#43742) wasm sample to use publish. * Fix NegotiateStream handling of EOF (#43739) In my refactoring of NegotiateStream to use async/await, I broke its handling of EOF, with it throwing an exception instead of returning 0. This fixes it to correctly handle EOF. * Free allocated buffer after UTF8 encode on FreeBSD (#43431) * Free allocated buffer after UTF8 encode on FreeBSD * Address PR feedback * [master] Update dependencies from mono/linker (#43698) [master] Update dependencies from mono/linker * Use non-generic TaskCompletionSource where possible in System.Threading.Channels (#40953) * Use non-generic TaskCompletionSource where possible * Apply suggestions from code review Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Stephen Toub <[email protected]> * Remove Common Extensions HashCodeCombiner (#43454) * Remove Common Extensions HashCodeCombiner Fix #33259 * Add unit tests for FilePatternMatch.GetHashCode. * Add link to Discord server (#43765) * Add link to Discord server * Update README.md Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Stephen Toub <[email protected]> * Suppress trim analysis warnings inside reflection implementation (#43594) See the Justification for each of the added suppressions. Also annotate GetMethodBody as potentially dangerous with trimming. Trimming can change IL of a method (can remove branches, remove local variables, change some isntructions, ...) - as such accessing the actual body of a method is potentially dangerous. * Fix issue #36999 (#43723) * Created two resources string for exceptions. Replaced static strings with resource. * Fixed formatting. Co-authored-by: Nikolay-parhimovich <> * Improve error message thrown by Microsoft.Extensions.Configuration.Json.JsonConfigurationFileParser when the root element of a JSON config file is not an object. (#42780) JSON configuration files must have an object as the root element. Previously, the exception message produced by Microsoft.Extensions.Configuration.Json.JsonConfigurationFileParser when a non-object root element was parsed was Error: Unsupported JSON token 'TOKEN_TYPE' was found. This error message was vague and caused confusion. This commit updates the error message to specifically mention that the root element must be an object. Fix dotnet/extensions#3543. * Fix null reload token for ConfigurationProvider (#43306) * Fix null reload token for ConfigurationProvider * Add not null assert * Setting `WebProxy.BypassList` to null throws (#40656) * Argument validation in WebProxy BypassList setter * Updated a failing test * Added tests for new BypassList behavior * Updated the tests for new BypassList behavior * Added AllowNull attribute on BypassList property * Update src/libraries/System.Net.WebProxy/src/System/Net/WebProxy.cs Co-authored-by: Stephen Toub <[email protected]> * Mono: Clang bin path not added to PATH for VS2019 builds. (#43505) Building full AOT using CI scripts won't work when VS2019 IDE is present (not falling back to VS2019 build tools). This happens due to a typo in setup-vs-msvcbuild-env.bat. Since CI doesn't install VS IDE, but using buildtools (where clan bin path is added to PATH) problem only shows up on local development environment using VS2019 IDE. Co-authored-by: lateralusX <[email protected]> * Block the runtime on ready to early bind breakpoints (#43260) * Block in ready * Remove hideWebDriver option and always do it * Rename attach method * Fix issue: ConventionsBuilder and open generic export with constructor dependencies (#43003) * Add new function to scan the open generic's constructors when the type is generic Co-authored-by: HuyLuong <[email protected]> * [w32socket] Turn WireGuard ENOKEY errno to WASANETUNREACH (#43734) When the destination IP on the packet doesn't match any WireGuard peer (such as if the peer is disconnected while the app is running), the sender may get an ENOKEY errno. This is mentioned in Section 3 "Send/Receive" of https://www.wireguard.com/papers/wireguard.pdf Fixes https://github.com/mono/mono/issues/20503 Co-authored-by: lambdageek <[email protected]> * improve link detection for WIFI on macOS (#43737) * improve link detection for WIFI on macOS * enable test back * correct condition * improve type detection for WIFI * Update src/libraries/Native/Unix/System.Native/pal_interfaceaddresses.c Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Stephen Toub <[email protected]> * Fix inconsistency of the 'CanRead' property after disposing an HTTP content stream (#43766) * Fix inconsistency of the 'CanRead' property after disposing an HTTP content stream * Update src/libraries/System.Net.Http/tests/FunctionalTests/RawConnectionStreamTest.cs Co-authored-by: Marie Píchová <[email protected]> Co-authored-by: Angelo Breuer <[email protected]> Co-authored-by: Marie Píchová <[email protected]> * [mono] Use lax Roslyn and analyzers settings for samples (#43769) * [mono] Use lax Roslyn and analyzers settings for samples The samples are often used for debugging runtime problems by modifying the code to reproduce issues. The code analyzers and warnings-as-errors add a papercut before a modified sample can run. * [mono] Make HelloWorld sample Makefile settings overridable allow MONO_CONFIG and MONO_ENV_OPTIONS to be set from the outside * [wasm][debugger] Fix check for already loaded assemblies (#43747) We send assembly loaded events to the proxy based off events from the debugger engine. And we check that it isn't an assembly that was already loaded. This check has a bug in computing the assembly name, from the filename, which caused these events to be sent even for already loaded assemblies. * Pass -target arm or -target arm64 to superpmi.exe when replaying in superpmi.py (#43784) * Pass -target (arm|arm64) argument to superpmi.exe when replaying for arm/arm64 in superpmi.py * Add the -target argument only in altjit in superpmi.py * Update Android instrumentation runner to be closer to upstream (#43775) The upstream instrumentation runners don't use runOnMainSync() in the onStart() method, update our runner to do the same and add a bit more logging. Also fixed a small typo in configure.cmake that I happened to notice. * Use function pointers for interop (Unix) (#43793) * [master] Update dependencies from dotnet/arcade Microsoft/vstest dotnet/llvm-project dotnet/runtime-assets mono/linker (#43768) [master] Update dependencies from dotnet/arcade Microsoft/vstest dotnet/llvm-project dotnet/runtime-assets mono/linker * Disable JIT counters if the JIT is disabled. (#43808) * Fix RHEL7 socket dispose hang, and extend coverage (#43409) Fix #42686 by doing a graceful close in case if the abortive connect(AF_UNSPEC) call fails on Linux, and improve couple of related tests: - Extend RetryHelper with an exception filter - Connect, Accept, Send, Receive, SendFile cancellation tests: make sure cancellation failures are not masked by RetryHelper (use exception filter) - Connect, Accept, Send, Receive cancellation tests: also test IPV6 and DualMode sockets * Fix to set the inner exception for ALC event (#43667) * Fix to set the inner exception for ALC event Removes the exception handling at CLRPrivBinderCoreCLR::BindAssemblyByName so that the inner exceptioncan be set when the default AssemblyLoadContext.Resolving handler throws * Fixing the test for alc.default * Enhanced the tests by customizing the exception type that will be thrown by the handlers * [w32process-win32] Implement System.Diagnostics.Process::MainWindowHandle. (#43724) L.A.Noire splash screen calls it to check if the game has opened its window, and stays forever visible if this is not implemented. Co-authored-by: rbernon <[email protected]> * Ensure IBC data is copied to the expected location (#43644) * [docs] Add area-System.Reflection-mono area owners (#43825) * CoreCLR runtime tests + Mono LLVM AOT on arm64 Linux (#41751) This change: - Adds new options to mono.proj: - MonoAOTEnableLLVM, which enables (or disables) building a Mono AOT cross compiler with LLVM; - MonoAOTLLVMDir, which specifies the path to a copy of LLVM suitable for building a Mono AOT cross compiler, and is optional; and - BuildMonoAOTCrossCompilerOnly, which allows building a Mono AOT cross compiler without building an associated full Mono runtime. - Changes offsets-tool.py's user interface slightly; '--include-prefix' is renamed to '--prefix' and may be specified multiple times. While this latter feature isn't necessary to build a Mono cross compiler today, because we don't use distribution-supplied cross-compilation headers on CI, it does make it easier to experiment with the offsets tool using arbitrary header layouts. For example, Debian's arm64 cross-compilation packages scatter useful header files across `/usr/lib/gcc-cross/aarch64-linux-gnu` and `/usr/aarch64-linux-gnu`. - Updates the docker image used for arm64 cross-compilation to a newer revision that includes libclang (see also https://github.com/dotnet/dotnet-buildtools-prereqs-docker/pull/375). - Adds some tests that currently fail to compile with Mono LLVM AOT to issues.targets. - Adds a Linux_arm64 LLVM AOT job to CI. Nothing particularly fancy is done to build the Mono LLVM AOT cross compiler; it is built as a step in the same job that also sends Linux_arm64 tests to Helix. * ServiceProcess Controller Refactor (#43797) 1. Simplified if checks. 2. Use expression property syntax. 3. Used using declaration. 4. Inlined out declaration. 5. Removed unwated unsafe modifier. 6. Removed redundant casting. * Enhance #43238 so that it covers DefineScope method group with the tests to reflect the behavior added in the PR. (#43790) * Bump emscripten to 2.0.6. (#43800) * Bump emscripten to 2.0.6. * Define HAVE_SYS_RANDOM_H on wasm, its not detected correctly. * Fixed bug in ReadOnlyDictionary's IDictionary.this[object] implementation. This method didn't adhere to IDictionary's contract to return null on a missing key. (#36926) * Add cancellable and AddressFamily-specific name resolution. (#33420) Add AddressFamily-specific name resolution and cancellation support for Windows. Resolves #939 * Setting value of enums didn't properly widen the value when setting (#43779) Use VerifierCorElementType instead of SignatureCorElementType to specify the element type of the target field - This will send the runtime down the path which can perform primitive widening * Merge PAL's _wcslwr into _wcslwr_s (#43265) In (non-palsuite) product code, `_wcslwr` is only used within PAL inside `_wcslwr_unsafe()` method, which is exposed as `_wcslwr_s` for PAL consumers. PR inlines the usage of `_wcslwr` in `_wcslwr_unsafe` and fixes up PAL tests. * [WIP][interp] Unify execution and valuetype stacks (#43681) Before this change, an InterpFrame contained 3 regions of data : args + locals, valuetype stack, execution stack. Each entry on the execution stack was a stackval structure. The space for valuetypes, was allocated separately, since they have various sizes. When pushing a valuetype on the stack, we allocated first the space for it on the vtstack and then pushed the address of the region on the execution stack. This change merges the execution stack with the valuetype stack, meaning we push now variable sized data on the execution stack. In order to keep track of the current stack location, whenever we push a type on stack, during transform phase, we also keep track of the offset where this value will reside on stack, as well as the size it occupies. All callsites need to be informed how much they need to pop the stack for the arguments. While called code can access this space normally (the args are special locals belonging to the frame and are accessed directly as such), external code needs a new mechanism to detect each argument for a given frame. This is achieved with the lazily initialized arg_offsets array on an InterpMethod. The method doesn't need to be compiled for this array to be correctly initialized. Why : - this simplifies handling of valuetypes, their storage follows the same rules as a normal objref/primitive type - removes the common use of the vt_sp variable. The compiler no longer needs to reserve it in a register during the switch loop, we no longer need to save it with each call. The sp and ip become now the only variables describing the execution state in a method. - the flow of the data on the execution stack is well behaved now (with the exception of a few opcodes that update directly based on the stack offset). We were using the vtstack for some magic storage (MINT_VTRESULT for example) - this makes it such that the stack offset of every value is easily known at compile time, making it possible to completely drop the execution stack approach, and have every opcode have a unique dreg and a list of sregs (which are mapped to a certain stack offset). This will enable more advanced optimizations during compile stage. Co-authored-by: BrzVlad <[email protected]> * Revert "Try re-enabling IBC on macOS. (#39801)" (#43839) This reverts commit 3a4298cf15379678c4d437a6554a1453706cc3b3. * Statically linking coreclr and clrjit in single file host (#43556) * Statically linking coreclr and clrjit in single file host. * setting g_hmodCoreCLR on Unix * System.Globalization.Native.lib must build with coreclr to be linkable with it * Always use system unwind libs on FREEBSD * no DllMain when coreclr is statically linked * Handle cases when coreclr configuration is different from libraries * Adding and using PAL_GetPalHostModule * simplify SslStream_StreamToStream_Alpn_NonMatchingProtocols_Fail test (#43625) * simplify SslStream_StreamToStream_Alpn_NonMatchingProtocols_Fail test * feedback from review * Improve annotations for XLinq methods taking params object[] (#43717) * Improve annotations for XLinq classes taking params object[] * annotate ref * address Jozkee's feedback * Fix XStreamingElement ctor to take nullable content * [RyuJIT] Propagate gtFlags in Vector.Create (#43578) Propagate GTF_CALL if needed in GT_LIST. Use gtNewListNode. Ignore test for Mono * [Portable thread pool] Don't spin-wait on semaphore when hill climbing stops a thread from processing work (#43840) * Add IDictionary_Generic_Tests test for multiple values with hash collisions (#43836) * Add IDictionary_Generic_Tests test for multiple values with hash collisions * Update src/libraries/System.Collections.Concurrent/tests/ConcurrentDictionary/ConcurrentDictionary.Generic.Tests.cs Co-authored-by: Eirik Tsarpalis <[email protected]> * Update src/libraries/System.Collections.Concurrent/tests/ConcurrentDictionary/ConcurrentDictionary.Generic.Tests.cs Co-authored-by: Eirik Tsarpalis <[email protected]> * Set TARGET_SIZEOF_VOID_P and SIZEOF_REGISTER correctly when cross compiling. (#43851) * Rewrite Socket.ConnectAsync for DNS with async/await (#43661) * Avoid several WildcardBindForConnectIfNecessary allocations on each connect * Rewrite Socket.ConnectAsync for DNS with async/await Rips out all of the APM code that was previously used to implement this and replaces it with {Value}Task-based async/await implementations. * add missing CBOR xmldocs (#43882) * RyuJIT: Fold Popcnt.PopCount with constant argument (#37836) Intrinsify BitOperations.PopCount for constant input * [RyuJIT] Fold "(X op C1) op C2" to "X op (C1 op C2)" for commutative operators (#43567) Fold "(X op C1) op C2" to "X op (C1 op C2)" * Fix covariant return type validation for canon parents (#43843) The return type validation was rejecting cases when the method being overriden had canon type in its generic arguments. This change fixes the problem by using parent method type instantiation for constructing the SigTypeContext in such case. It also adds a regression test. * Reimplement Socket.Begin/EndSend/Receive on Send/ReceiveAsync (#43886) * Remove a volatile access from Task.Id (#43891) * Add info about how named mutexes work on Unix into code from the orginal PR (#43161) * Add exception case xml comment for ExecutionContext.Restore * Add some info about how named mutexes work on Unixes into code from the PR * Fix test for 32-bit platforms (#43888) - Fixed an assertion failure. `WorkerCounter` shouldn't be used in the native thread pool implementation when the portable thread pool is enabled (all the counts will be zero), fixed a missed case in `GetAvailableThreads`. - The native implementation uses a smaller max default worker thread count by default on 32-bit platforms, allowing configured values to go beyond that. Fixed the portable thread pool implementation to do similar, instead of limiting the max including for configured values. * Make runtime tests run with Android (#42683) * Prototype for runtime tests running with Android * Conditionally collect app dependencies * Switch android sample from publish to build * Switch Android sample from publish to build * Modify AndroidTestRunner * Only build and test one test * Clean up some changes * Update new run test script path and update AndroidTestRunner to use files under CoreRoot * Add Helix configuration for Android_x64 * Disable test which replies on coreclr System.Private.CoreLib.dll * Fixed format and removed irrelevant parameters * Rmoved unused parameters and fixed comment * Revert my AndroidAppBuilder related changes to prepare for…

* Fix inconsistency of the 'CanRead' property after disposing an HTTP content stream (#43766) * Fix inconsistency of the 'CanRead' property after disposing an HTTP content stream * Update src/libraries/System.Net.Http/tests/FunctionalTests/RawConnectionStreamTest.cs Co-authored-by: Marie Píchová <[email protected]> Co-authored-by: Angelo Breuer <[email protected]> Co-authored-by: Marie Píchová <[email protected]> * [mono] Use lax Roslyn and analyzers settings for samples (#43769) * [mono] Use lax Roslyn and analyzers settings for samples The samples are often used for debugging runtime problems by modifying the code to reproduce issues. The code analyzers and warnings-as-errors add a papercut before a modified sample can run. * [mono] Make HelloWorld sample Makefile settings overridable allow MONO_CONFIG and MONO_ENV_OPTIONS to be set from the outside * [wasm][debugger] Fix check for already loaded assemblies (#43747) We send assembly loaded events to the proxy based off events from the debugger engine. And we check that it isn't an assembly that was already loaded. This check has a bug in computing the assembly name, from the filename, which caused these events to be sent even for already loaded assemblies. * Pass -target arm or -target arm64 to superpmi.exe when replaying in superpmi.py (#43784) * Pass -target (arm|arm64) argument to superpmi.exe when replaying for arm/arm64 in superpmi.py * Add the -target argument only in altjit in superpmi.py * Update Android instrumentation runner to be closer to upstream (#43775) The upstream instrumentation runners don't use runOnMainSync() in the onStart() method, update our runner to do the same and add a bit more logging. Also fixed a small typo in configure.cmake that I happened to notice. * Use function pointers for interop (Unix) (#43793) * [master] Update dependencies from dotnet/arcade Microsoft/vstest dotnet/llvm-project dotnet/runtime-assets mono/linker (#43768) [master] Update dependencies from dotnet/arcade Microsoft/vstest dotnet/llvm-project dotnet/runtime-assets mono/linker * Disable JIT counters if the JIT is disabled. (#43808) * Fix RHEL7 socket dispose hang, and extend coverage (#43409) Fix #42686 by doing a graceful close in case if the abortive connect(AF_UNSPEC) call fails on Linux, and improve couple of related tests: - Extend RetryHelper with an exception filter - Connect, Accept, Send, Receive, SendFile cancellation tests: make sure cancellation failures are not masked by RetryHelper (use exception filter) - Connect, Accept, Send, Receive cancellation tests: also test IPV6 and DualMode sockets * Fix to set the inner exception for ALC event (#43667) * Fix to set the inner exception for ALC event Removes the exception handling at CLRPrivBinderCoreCLR::BindAssemblyByName so that the inner exceptioncan be set when the default AssemblyLoadContext.Resolving handler throws * Fixing the test for alc.default * Enhanced the tests by customizing the exception type that will be thrown by the handlers * [w32process-win32] Implement System.Diagnostics.Process::MainWindowHandle. (#43724) L.A.Noire splash screen calls it to check if the game has opened its window, and stays forever visible if this is not implemented. Co-authored-by: rbernon <[email protected]> * Ensure IBC data is copied to the expected location (#43644) * [docs] Add area-System.Reflection-mono area owners (#43825) * CoreCLR runtime tests + Mono LLVM AOT on arm64 Linux (#41751) This change: - Adds new options to mono.proj: - MonoAOTEnableLLVM, which enables (or disables) building a Mono AOT cross compiler with LLVM; - MonoAOTLLVMDir, which specifies the path to a copy of LLVM suitable for building a Mono AOT cross compiler, and is optional; and - BuildMonoAOTCrossCompilerOnly, which allows building a Mono AOT cross compiler without building an associated full Mono runtime. - Changes offsets-tool.py's user interface slightly; '--include-prefix' is renamed to '--prefix' and may be specified multiple times. While this latter feature isn't necessary to build a Mono cross compiler today, because we don't use distribution-supplied cross-compilation headers on CI, it does make it easier to experiment with the offsets tool using arbitrary header layouts. For example, Debian's arm64 cross-compilation packages scatter useful header files across `/usr/lib/gcc-cross/aarch64-linux-gnu` and `/usr/aarch64-linux-gnu`. - Updates the docker image used for arm64 cross-compilation to a newer revision that includes libclang (see also https://github.com/dotnet/dotnet-buildtools-prereqs-docker/pull/375). - Adds some tests that currently fail to compile with Mono LLVM AOT to issues.targets. - Adds a Linux_arm64 LLVM AOT job to CI. Nothing particularly fancy is done to build the Mono LLVM AOT cross compiler; it is built as a step in the same job that also sends Linux_arm64 tests to Helix. * ServiceProcess Controller Refactor (#43797) 1. Simplified if checks. 2. Use expression property syntax. 3. Used using declaration. 4. Inlined out declaration. 5. Removed unwated unsafe modifier. 6. Removed redundant casting. * Enhance #43238 so that it covers DefineScope method group with the tests to reflect the behavior added in the PR. (#43790) * Bump emscripten to 2.0.6. (#43800) * Bump emscripten to 2.0.6. * Define HAVE_SYS_RANDOM_H on wasm, its not detected correctly. * Fixed bug in ReadOnlyDictionary's IDictionary.this[object] implementation. This method didn't adhere to IDictionary's contract to return null on a missing key. (#36926) * Add cancellable and AddressFamily-specific name resolution. (#33420) Add AddressFamily-specific name resolution and cancellation support for Windows. Resolves #939 * Setting value of enums didn't properly widen the value when setting (#43779) Use VerifierCorElementType instead of SignatureCorElementType to specify the element type of the target field - This will send the runtime down the path which can perform primitive widening * Merge PAL's _wcslwr into _wcslwr_s (#43265) In (non-palsuite) product code, `_wcslwr` is only used within PAL inside `_wcslwr_unsafe()` method, which is exposed as `_wcslwr_s` for PAL consumers. PR inlines the usage of `_wcslwr` in `_wcslwr_unsafe` and fixes up PAL tests. * [WIP][interp] Unify execution and valuetype stacks (#43681) Before this change, an InterpFrame contained 3 regions of data : args + locals, valuetype stack, execution stack. Each entry on the execution stack was a stackval structure. The space for valuetypes, was allocated separately, since they have various sizes. When pushing a valuetype on the stack, we allocated first the space for it on the vtstack and then pushed the address of the region on the execution stack. This change merges the execution stack with the valuetype stack, meaning we push now variable sized data on the execution stack. In order to keep track of the current stack location, whenever we push a type on stack, during transform phase, we also keep track of the offset where this value will reside on stack, as well as the size it occupies. All callsites need to be informed how much they need to pop the stack for the arguments. While called code can access this space normally (the args are special locals belonging to the frame and are accessed directly as such), external code needs a new mechanism to detect each argument for a given frame. This is achieved with the lazily initialized arg_offsets array on an InterpMethod. The method doesn't need to be compiled for this array to be correctly initialized. Why : - this simplifies handling of valuetypes, their storage follows the same rules as a normal objref/primitive type - removes the common use of the vt_sp variable. The compiler no longer needs to reserve it in a register during the switch loop, we no longer need to save it with each call. The sp and ip become now the only variables describing the execution state in a method. - the flow of the data on the execution stack is well behaved now (with the exception of a few opcodes that update directly based on the stack offset). We were using the vtstack for some magic storage (MINT_VTRESULT for example) - this makes it such that the stack offset of every value is easily known at compile time, making it possible to completely drop the execution stack approach, and have every opcode have a unique dreg and a list of sregs (which are mapped to a certain stack offset). This will enable more advanced optimizations during compile stage. Co-authored-by: BrzVlad <[email protected]> * Revert "Try re-enabling IBC on macOS. (#39801)" (#43839) This reverts commit 3a4298cf15379678c4d437a6554a1453706cc3b3. * Statically linking coreclr and clrjit in single file host (#43556) * Statically linking coreclr and clrjit in single file host. * setting g_hmodCoreCLR on Unix * System.Globalization.Native.lib must build with coreclr to be linkable with it * Always use system unwind libs on FREEBSD * no DllMain when coreclr is statically linked * Handle cases when coreclr configuration is different from libraries * Adding and using PAL_GetPalHostModule * simplify SslStream_StreamToStream_Alpn_NonMatchingProtocols_Fail test (#43625) * simplify SslStream_StreamToStream_Alpn_NonMatchingProtocols_Fail test * feedback from review * Improve annotations for XLinq methods taking params object[] (#43717) * Improve annotations for XLinq classes taking params object[] * annotate ref * address Jozkee's feedback * Fix XStreamingElement ctor to take nullable content * [RyuJIT] Propagate gtFlags in Vector.Create (#43578) Propagate GTF_CALL if needed in GT_LIST. Use gtNewListNode. Ignore test for Mono * [Portable thread pool] Don't spin-wait on semaphore when hill climbing stops a thread from processing work (#43840) * Add IDictionary_Generic_Tests test for multiple values with hash collisions (#43836) * Add IDictionary_Generic_Tests test for multiple values with hash collisions * Update src/libraries/System.Collections.Concurrent/tests/ConcurrentDictionary/ConcurrentDictionary.Generic.Tests.cs Co-authored-by: Eirik Tsarpalis <[email protected]> * Update src/libraries/System.Collections.Concurrent/tests/ConcurrentDictionary/ConcurrentDictionary.Generic.Tests.cs Co-authored-by: Eirik Tsarpalis <[email protected]> * Set TARGET_SIZEOF_VOID_P and SIZEOF_REGISTER correctly when cross compiling. (#43851) * Rewrite Socket.ConnectAsync for DNS with async/await (#43661) * Avoid several WildcardBindForConnectIfNecessary allocations on each connect * Rewrite Socket.ConnectAsync for DNS with async/await Rips out all of the APM code that was previously used to implement this and replaces it with {Value}Task-based async/await implementations. * add missing CBOR xmldocs (#43882) * RyuJIT: Fold Popcnt.PopCount with constant argument (#37836) Intrinsify BitOperations.PopCount for constant input * [RyuJIT] Fold "(X op C1) op C2" to "X op (C1 op C2)" for commutative operators (#43567) Fold "(X op C1) op C2" to "X op (C1 op C2)" * Fix covariant return type validation for canon parents (#43843) The return type validation was rejecting cases when the method being overriden had canon type in its generic arguments. This change fixes the problem by using parent method type instantiation for constructing the SigTypeContext in such case. It also adds a regression test. * Reimplement Socket.Begin/EndSend/Receive on Send/ReceiveAsync (#43886) * Remove a volatile access from Task.Id (#43891) * Add info about how named mutexes work on Unix into code from the orginal PR (#43161) * Add exception case xml comment for ExecutionContext.Restore * Add some info about how named mutexes work on Unixes into code from the PR * Fix test for 32-bit platforms (#43888) - Fixed an assertion failure. `WorkerCounter` shouldn't be used in the native thread pool implementation when the portable thread pool is enabled (all the counts will be zero), fixed a missed case in `GetAvailableThreads`. - The native implementation uses a smaller max default worker thread count by default on 32-bit platforms, allowing configured values to go beyond that. Fixed the portable thread pool implementation to do similar, instead of limiting the max including for configured values. * Make runtime tests run with Android (#42683) * Prototype for runtime tests running with Android * Conditionally collect app dependencies * Switch android sample from publish to build * Switch Android sample from publish to build * Modify AndroidTestRunner * Only build and test one test * Clean up some changes * Update new run test script path and update AndroidTestRunner to use files under CoreRoot * Add Helix configuration for Android_x64 * Disable test which replies on coreclr System.Private.CoreLib.dll * Fixed format and removed irrelevant parameters * Rmoved unused parameters and fixed comment * Revert my AndroidAppBuilder related changes to prepare for getting Egor's AndroidAppBuilder change * Fixed rebase and merge error * Update AndroidTestRunner * Use the AndroidAppBuilder in the publish folder * Update runtime test Android App creation command * Temporarily disable all jobs except for the Android one * Fix up job running config * Fixed AndroidTestRunner * Get Main function return code after running android app * Fix for skipping building native tests * Fix helix project path, due to recent coreclr test related files relocation. * Fix android app running commands * Add missing helixType * Update HelixRuntimeRid * Fix inconsistency * Create Android app during build preiod. * Adjust the way to run tests * Temporary hack before AndroidAppBuilder could accept multiple assemblies * Use xharness instead of adb to run tests on Androids and disable cmake for Android. * Cleanup unnecessary changes * Include xharness Cli * Use the right xharness command * Copy runtime assemblies, since the copy logic got removed from AndroidAppBuilder * Save Android apps in CORE_ROOT dir and adjust BuildAndroidApp target due to rebase from master * Minor cleanup * Enable all runtime tests and disable tests caused build failure * Copy over test helper assemblies and add failing tests to issues.targets * Remove redundant comment * Revert previous commit * Only sending apk files to helix machine * Delete OutputDir after finishing building the app * Copy whole Core_Root dir to helix machine. * Extend timeout threshold * Extend timeout threshold for each test and each test collection * Display Android app running logs when tests fail * Android tests are required to run in sequential * Check if log file exists before using it. * Exclude two out-of-memory tests * Fixed merge error * Shorten the timeout threshold * Enable all the lanes * Fix indentation * Fixed a few issues from review feedback * Update src/tests/run.proj Co-authored-by: Alexander Köplinger <[email protected]> * Only bail out from the bash script when erroring out for andriod * Extend timeout threshold, since the run could take a long time or a short time on CI. * Fixed merge error * Filter out tests using more memory than allowed by helix machine Co-authored-by: Alexander Köplinger <[email protected]> * [wasm] Add backward compatibility file to timezone data compilation (#43786) * Add backward file from iana to handle backward compatibility in tz data * add tests to check for backwards compatibility * Some refactoring of thread suspension. (#34666) * m_dwForbidSuspendThread * Remove some fibers stuff * g_TrapReturningThreads * g_pGCSuspendEvent * m_pThreadAttemptingSuspendForGC * refactor SuspendRuntime into just one loop * Suspend iteration after updating hijacks should "observeOnly" - no point to update hijacks again right away. * avoid retrying to suspend threads that are already in redirected state. * PR feedback * "premptive" - common typo apparently * missing profiler notification on suspend end and `DisableStressHeap` stuff * indentation and infinite loop style * Removed GTF_DONT_CSE flag on `g_TrapReturningThreads` load in GCPoll and added comment explaining why. * reverted an accidentally removed assert * removed SWITCHOUT_HANDLE_VALUE entirely - not different from INVALID_HANDLE_VALUE now * some cleanups and more comments when non-GC suspension yields to GC * Few tweaks for the concerns discussed in the PR * On Server GC `pCurThread` is NULL. Need to fix an ASSERT to account for the possibility of NULL. * Add link to Discord server to README.md (#43926) * Throw when reader is completed with an exception (#43776) * Disable qcalls on wasm. Treat them as normal pinvokes. (#43798) * Disable qcalls on wasm. Treat them as normal pinvokes. This is needed because they are stored in a separate table, so they cannot be linked out the same way as pinvokes/icalls. * Update tools-local/tasks/mobile.tasks/WasmAppBuilder/PInvokeTableGenerator.cs Co-authored-by: Ryan Lucia <[email protected]> * Update tools-local/tasks/mobile.tasks/WasmAppBuilder/PInvokeTableGenerator.cs Co-authored-by: Ryan Lucia <[email protected]> Co-authored-by: Ryan Lucia <[email protected]> * Change Task.FromResult to use same task cache as async methods (#43894) * Change Task.FromResult to use same task cache as async methods Task.FromResult today always creates a new task. This leads developers that are aware of this to then create their own caches for common values, typically 0, true, and false, to avoid task allocations for those common values. But we already have such values cached internally, used for async method return values. We can just use the same cache for Task.FromResult. * Address PR feedback * Add initial set of conformance tests for Stream (#43834) * Fix invalid assert in StreamBuffer This can be hit by calling code and thus shouldn't be an assert. As an assert it prevents testing. * Add ConnectedStreams.CreateUnidirectional * Add initial set of Stream conformance tests These primarily focus on "connected streams", ones that can be arranged to communicate with each other in a producer/consumer pattern or as a bidirectional communication mechanism, e.g. NetworkStream, PipeStream, SslStream wrapped around a NetworkStream, FileStream created around pipes, etc. Later we can add more tests focused on standalone streams, e.g. FileStream created for an on-disk file, MemoryStream, etc. * Add ConnectedStreams tests These are currently helpers used by many other tests. At some point they could become public API as well. * Fix NetworkStream argument names Technically a breaking change, but the current divergence from the base Stream class is a bug, and bringing them into sync means argument exceptions that emerge from the derived type make sense when used via the base type, as well as then being able to use shared validation logic across all streams (subsequent to these changes). * Add stream conformance tests for NetworkStream * Add stream conformance tests for QuicStream * Fix several CryptoStream behaviors 1. Flushing a stream that wraps another stream for writing should always flush that underlying stream, even if no additional data was written as part of the flush. 2. Argument validation should validate buffers are not null rather than null ref'ing on a null buffer. 3. Checks for the CryptoStream mode should come after argument validation. * Add stream conformance tests for CryptoStream * Fix FileStream argument names Technically a breaking change, but the current divergence from the base Stream class is a bug, and bringing them into sync means argument exceptions that emerge from the derived type make sense when used via the base type, as well as then being able to use shared validation logic across all streams (subsequent to these changes). * Fix BufferedStream argument names Technically a breaking change, but the current divergence from the base Stream class is a bug, and bringing them into sync means argument exceptions that emerge from the derived type make sense when used via the base type, as well as then being able to use shared validation logic across all streams (subsequent to these changes). * Add stream conformance tests for FileStream Specifically when used in a connected fashion, wrapped around an anonymous or named pipe. * Add stream conformance tests for BufferedStream Specifically when used in a connected fashion, wrapped around some other connected stream. * Add stream conformance tests for SslStream and NegotiateStream * Fix PipeStream.Flush to not fail on readable streams Consumers may expect Stream.Flush to be a nop if the stream is readable only, but PipeStream.Flush is throwing in that case. Stop doing that. * Add stream conformance tests for PipeStream * Fix several BrotliStream behaviors 1. When passed a null buffer, it's throwing an exception with the argument name "array", even though the parameter's name is "buffer". 2. Even if there's no data written as part of the Flush{Async} call on a writeable stream, it should be calling flush on the wrapped stream. * Fix several DeflateStream (and friends) issues 1. DeflateStream.Flush{Async} when writing needs to always flush the underlying stream, even if there's no data written as part of the flush call itself. 2. Several byte[] array arguments should be byte[] buffer. Technically a breaking change, but the current divergence from the base Stream class is a bug, and bringing them into sync means argument exceptions that emerge from the derived type make sense when used via the base type, as well as then being able to use shared validation logic across all streams (subsequent to these changes). 3. DeflateStream.EndRead/Write needs to do additional state validation. 4. Not a bug, but simplify ReadAsync to match the sync Read implementation flow. * Add stream conformance tests for Deflate/ZLib/GZip/BrotliStream * Fix PipeReader/WriterStream argument validation * Add stream conformance tests for PipeReader/WriterStream * Remove erroneous asserts from Stream.ReadWriteTask * Address PR feedback * Fix a few tests in CI * Add interpreter libraries to iOS device runtime pack (#43944) Fixes https://github.com/dotnet/runtime/issues/43943 * [mono] Fix LoadAssemblyRaw to not pin or copy (#43830) * [mono] Fix LoadAssemblyRaw to not pin or copy Copying of the assembly and symbols happens later, in `mono_alc_load_raw_bytes`, and the pinning is no longer necessary so long as we have the handle. * Handle null symbol store * Respect reloadOnChange in UserSecrets in DefaultBuilder (#43807) * Respect reloadOnChange in UserSecrets in DefaultBuilder * Replace net6.0 TargetFramework Equality Conditions with msbuild intrinsic functions (#43857) * using msbuild intrinsic functions to get tfm identifier and tfv version * Update src/libraries/Microsoft.Win32.SystemEvents/src/Microsoft.Win32.SystemEvents.csproj Co-authored-by: Viktor Hofer <[email protected]> * Update build-native.proj * Update build-native.proj Co-authored-by: Viktor Hofer <[email protected]> * Enable the portable thread pool by default in coreclr (#43841) * Fix DI DisposeAsync being called twice (#43803) * Fix DI DisposeAsync being called twice * [master] Update dependencies from mono/linker dotnet/arcade dotnet/xharness dotnet/llvm-project dotnet/runtime-assets dotnet/icu (#43822) * Update dependencies from https://github.com/mono/linker build 20201026.1 Microsoft.NET.ILLink.Tasks From Version 6.0.0-alpha.1.20525.1 -> To Version 6.0.0-alpha.1.20526.1 * Update dependencies from https://github.com/dotnet/arcade build 20201022.2 Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.ApiCompat From Version 6.0.0-beta.20520.8 -> To Version 6.0.0-beta.20522.2 * Update dependencies from https://github.com/dotnet/icu build 20201026.1 Microsoft.NETCore.Runtime.ICU.Transport From Version 6.0.0-alpha.1.20512.1 -> To Version 6.0.0-alpha.1.20526.1 * Update dependencies from https://github.com/mono/linker build 20201026.2 Microsoft.NET.ILLink.Tasks From Version 6.0.0-alpha.1.20525.1 -> To Version 6.0.0-alpha.1.20526.2 * Update dependencies from https://github.com/mono/linker build 20201027.1 Microsoft.NET.ILLink.Tasks From Version 6.0.0-alpha.1.20525.1 -> To Version 6.0.0-alpha.1.20527.1 * Update dependencies from https://github.com/dotnet/arcade build 20201025.1 Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.ApiCompat From Version 6.0.0-beta.20520.8 -> To Version 6.0.0-beta.20525.1 * Update dependencies from https://github.com/dotnet/xharness build 20201027.1 Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 1.0.0-prerelease.20519.2 -> To Version 1.0.0-prerelease.20527.1 * Update dependencies from https://github.com/dotnet/llvm-project build 20201026.1 runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.win-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.osx.10.12-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.linux-x64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk , runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Tools , runtime.linux-arm64.Microsoft.NETCore.Runtime.Mono.LLVM.Sdk From Version 9.0.1-alpha.1.20523.1 -> To Version 9.0.1-alpha.1.20526.1 * Update dependencies from https://github.com/dotnet/runtime-assets build 20201026.1 System.ComponentModel.TypeConverter.TestData , System.Drawing.Common.TestData , System.IO.Compression.TestData , System.IO.Packaging.TestData , System.Net.TestData , System.Private.Runtime.UnicodeData , System.Runtime.TimeZoneData , System.Security.Cryptography.X509Certificates.TestData , System.Windows.Extensions.TestData From Version 5.0.0-beta.20522.4 -> To Version 5.0.0-beta.20526.1 * Update dependencies from https://github.com/mono/linker build 20201027.2 Microsoft.NET.ILLink.Tasks From Version 6.0.0-alpha.1.20525.1 -> To Version 6.0.0-alpha.1.20527.2 * Update dependencies from https://github.com/dotnet/arcade build 20201026.10 Microsoft.DotNet.Build.Tasks.Feed , Microsoft.DotNet.Build.Tasks.Packaging , Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk , Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk , Microsoft.DotNet.CodeAnalysis , Microsoft.DotNet.GenAPI , Microsoft.DotNet.GenFacades , Microsoft.DotNet.XUnitExtensions , Microsoft.DotNet.Arcade.Sdk , Microsoft.DotNet.Helix.Sdk , Microsoft.DotNet.RemoteExecutor , Microsoft.DotNet.VersionTools.Tasks , Microsoft.DotNet.XUnitConsoleRunner , Microsoft.DotNet.ApiCompat From Version 6.0.0-beta.20520.8 -> To Version 6.0.0-beta.20526.10 * Update dependencies from https://github.com/dotnet/runtime-assets build 20201027.1 System.ComponentModel.TypeConverter.TestData , System.Drawing.Common.TestData , System.IO.Compression.TestData , System.IO.Packaging.TestData , System.Net.TestData , System.Private.Runtime.UnicodeData , System.Runtime.TimeZoneData , System.Security.Cryptography.X509Certificates.TestData , System.Windows.Extensions.TestData From Version 5.0.0-beta.20522.4 -> To Version 5.0.0-beta.20527.1 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Add static libraries for System.Native back to mobile runtime packs (#43987) We need it for various scenarios there. It was removed in https://github.com/dotnet/runtime/pull/41966. Fixes https://github.com/dotnet/runtime/issues/43949 * [interp] return false for IsLittleEndian on BE (#43946) To help those who maintain Mono for BE. Also, it allows interpreter to fold branches since now IsLittleEndian is converted into `LDC.I4.1` always on LE. Example: ```csharp static void Validate() { if (!BitConverter.IsLittleEndian) throw new PlatformNotSupportedException(); } ``` Current output for `MONO_VERBOSE_METHOD=Validate` on both LE/BE: ``` IR_0000: ldsfld.u1 0 IR_0003: brtrue.i4.s IR_000b IR_0005: newobj_fast 3 IR_000a: throw IR_000b: ret.void ``` New output on LE: ``` IR_0000: ret.void ``` * Add back passing of --build-id option to Mono runtime build (#43993) This got lost as part of CMake cleanup in https://github.com/dotnet/runtime/pull/43678 but it is actually important for some publishing infrastructure: https://github.com/dotnet/arcade/issues/6464 * Bump xharness (#43996) Includes a potential fix for https://github.com/dotnet/runtime/issues/43983 * Clarify string.IndexOf rules in code comments (#43999) * Avoid disabling assert messages in debug configuration on wasm to help debugging. (#44004) * Update ServiceProviderServiceExtensions.cs (#43974) Use pattern matching, rather than `as` cast and `null` check. * Remove redundant lower function (#44010) * Fix #44004, disable assertion messages in release mode instead of debug. (#44023) * Fix #44004, disable assertion messages in release mode instead of debug. * Remove obsolete 'full_messages' enable-minimal option. * Disable the 'problematic assembly' code on .net core. (#44022) * Fix loading component when application was started with `dotnet` instead of apphost (#43962) * Don't parse arguments in hostpolicy until necessary * Explicit error on invalid startup info in hostfxr * Update tests * Support loading public-only PEM X.509 certificates Create an overload of CreateFromFile that accepts just the PEM-encoded X.509 certificate that does not contain a private key. * Update coding-style.md * remove unnecessary bound check (#42431) now that all primitive types are supported (thanks to #29000), we can remove custom bound checks. * Use jitstd::sort instead of qsort in jit and gcinfo (#43923) Current use of qsort interferes with cross compilation. The qsort algorithm is quite different between Windows and Linux, thus causing cross compilation comparison failures while testing. This replaces qsort with the use of jitstd::sort which provides a sort which is consistent between all platforms. (Particularly in Windows where CFG has costs for function pointer usage.) With a homegrown sort performance test, there is about 15-30% performance win with this sort over qsort. (However as these tests do not resemble the behavior of these sorts in the JIT I do not believe they are spectacularly relevant.) There is a size cost of about 4KB-6KB for the templatized routines. I was unable to measure a performance difference for the JIT itself. (Crossgen is very slightly faster in my testing with the change, but I do not believe the difference is statistically meaningful.) * Remove some unnecessary List-related allocation from EventSource (#44026) * Use BinaryPrimitives where possible (#43474) * Update dependencies from https://github.com/dotnet/arcade build 20201027.10 (#44020) [master] Update dependencies from dotnet/arcade * Enable automated SPMI collection for outerloop tests (#43773) * wip * wip * Parameterized SPMI collection * move the condition * change to variables * Update displayName * Change the format * Revert "Change the format" This reverts commit 292ba21c95026feae7528be449c99a0f20b4b068. * Use variables.collectionName * fix displayName * use variables.collectionName for displayName * print * Use name * fix condition * Create 2 yml files * Download test artifacts for tests collection * Attempt to have multiple jobs * Back to superpmi.yml * Add missing parameter * Use CollectionName at other places * Use collectionName for artifacts, increase timeout for test * try to merge whatever mch files we got * Fixes - increase the timeout for tests to 8 hours - ignore file error in superpmi.py - revert timeout setting from superpmi.proj * fix the .mch file name * fix typo * Add Stream argument validation helpers and use throughout dotnet/runtime (#43968) * Add Stream argument validation helpers and use throughout dotnet/runtime * Delete unnecessary code in Stream Stream.Null was using BlockingBegin/EndRead/Write, but the operations are nops (they don't even do argument validation), so the functionality was unnecessary complication. We can just use TaskToApm instead, with already completed tasks, and then also delete the SynchronousAsyncResult that was used by these operations. Also, the Begin/EndRead/Write methods were checking CanRead/Write and throwing if they return false, but this is a private sealed type with CanRead/Write hardcoded to return true! Delete. Finally added consistency across Stream.Null to checking for cancellation. * Clean up some style on Stream Also consolidate a few resources and use a few existing ThrowHelpers where appropriate. * Fix a few issues, and remove duplicative tests missed in conformance tests rollout * Address PR feedback * Remove unused ifdef logic in ODBC (#44042) * Address JsonElement doc and test feedback (#43832) * [cmake] Add -DCHECKED_BUILD=args support, and check getters in Debug (#44024) * [cmake] Add -DCHECKED_BUILD=args support, and check getters in Debug Add support for additional compile-time and runtime checks to cmake. Also for Debug builds, turn on the compile-time "private types" check that verifies that getters are being used to access the fields of certain structs. * [interp] Use getters * remove unused stub * Implement the reuse of duplicate constant values (#43899) * Implement the reuse of duplicate constant value - Duplicate values will now be reused in the constants area - Added MIN_DATA_ALIGN and MAX_DATA_ALIGN constants - Added dsDataType to record the type of the constant - Print the floating point values in the constants area - Changed 'alignment' and 'cnsSize' parameters to be unsigned instead of UNATIVE_OFFSET - Added method emitDataGenFind to search and return a duplicate constant value * Correctly place declaration of curOffs * Code review feedback * Carol's feedback * Feedback from Tanner Gooding Allow binary bit pattern matches of constants * Fix build warning/error * Jit Format * Fix gsharedvt constrained calls to Object.GetHashCode () which was broken by #43729. (#44047) Fixes https://github.com/dotnet/runtime/issues/44000. * Enable unused-result warning for PAL (#43395) Delete the remaining dead variables and methods from PAL, that were highlighted by gcc's dead code analysis, and enable `unused-result` warning for PAL. * Set number handling for properties read after deserializing ctor params (#43829) * Set number handling for properties read after deserializing ctor params * Use DeserializeWrapper pattern for tests * Set default value for value-type ctor params properly (#43831) * Set default value for value-type ctor params properly * Address review feedback * Delete area-Tizen * EqualityComparer IndexOf to shared (#44065) * Clean up debugger debug logging and wasm trace logging (#43960) * Clean up debugger debug logging and wasm trace logging * Update src/mono/mono/mini/mini-wasm-debugger.c Co-authored-by: Ryan Lucia <[email protected]> * Add area owners for System.Formats (#44038) * Asn1 * Cbor * Align last pipe, add missing pipes. * Quic * Additional owners for Asn1 and Cbor Co-authored-by: carlossanlop <[email protected]> * [Interp] Create an intrinsic for Interlocked.Exchange for int64 (#43893) * Add an instrinsic for Interlocked.Exchange * Hacks to help with development * Remove one more layer of function call and revert local hacks * Revert redundant format change * make interlocked_mutex a global variable * Update opcode name * Resolve reviewer feedbacks * Add a pair of braces. * Optimize String.Replace(string, string) (#44088) We can significantly improve its throughput for a few scenarios: - If both oldValue and newValue are single characters in the form of strings, we can just delegate to the Replace(char, char) overload, which is much faster. - If oldValue is a single character but newValue isn't, we can use IndexOf to find it, rather than an open-coded loop, making search times much faster for reasonably sized strings. - If oldValue is multiple characters and the string being searched for isn't super frequent (e.g. doesn't repeat every few characters), we can significantly speed up throughput by using IndexOf to search for the whole string. For example, replacing "\r\n" with "\n" in the contents of a typical file. This does come at a measurable cost when the oldValue is really common and tightly packed, e.g. searching for "aa" in "aaaaaaaaaaaaaaaaaa", so we can decide whether the tradeoff is the right one. * Disable PollingEventFires test (#44096) * Fix IL interpreter build on GCC. (#44049) * Minor update to function signatures for RISCV64 architecture (#43862) I don't believe this fixes any existing issue in the tracker, but I'm trying to build mono with dotnet-core on riscv from git and ran into this problem when compiling. There may end up being other changes but I won't know until my VM is done actually building mono. The changes right now are incredibly minor, just updating some function signatures to match the current rest of the system.  Co-authored-by: simcop2387 <[email protected]> * Remove WinRT leftovers from mono's CoreLib (#44102) * [master] Update dependencies from dotnet/arcade dotnet/llvm-project dotnet/xharness (#44062) [master] Update dependencies from dotnet/arcade dotnet/llvm-project dotnet/xharness * Fix the typo (#44075) * Avoid linking against libicudata.a, its not needed. (#44105) * Delete unnecessary seek from SetLengthInternal on Unix (#44097) Also, improve test coverage for FileStream.SetLength * Add optimized String.Join for IList<string> (#44032) * Support IList<string> * Revert API change * Feedback * Use ReadOnlySpan * Feedback * Fix comment * address feedback * Add test for List<string> optimization * Remove StartIndex and Count in core * Update src/libraries/System.Private.CoreLib/src/System/String.Manipulation.cs Co-authored-by: Stephen Toub <[email protected]> * Fix type generator test failures (#44041) - Fix issue where method token referred to derived type but method was actually defined on base type - Previously the methods did not carry enough state to determine when to emit the owner type of the method, and which exact type was the owning type. The new logic computes it from the token in the presence of a proper instantiation context, which allows for correct operation. - Fix issue where constrained dispatch on a method of a valuetype would not put in the correct owner type - Issue fixed by determining if the token resolves to a method on the same type as the eventual target method, or if it needs to have a specific owning type specified In general these issues where both caused by confusion around exactly the correct owning type, and it turned out that computation could not be computed within the signature emitter code, but instead needed to be computed in the JIT at point of use. Fortunately, we had a structure `MethodWithToken` that is used in these (and only these situations). Finally, I also made a pass through the emitter and related logic to remove various band-aids that had built up over the last few years to make all the tests and applications pass. I believe that the new logic should be correct in the general case. Bonus tweak... Use parallelism when compiling the framework composite images with crossgen2, and fix bug in composite image generation where mangled symbol names might conflict. Fixes #43466 and fixes #43467 * Remove last remnants of IExpando support. (#44122) * Remove unsafe code from String.Join (#44126) * Remove unsafe code from String.Join - Use a span for the separator to share the same code across a char separator and string separator, rather than using pointers. - Consolidate argument validation for the value/startIndex/count overloads into one shared helper - Change indexing to use standard 0 to values.Length for loop to eliminate bounds checking on span accesses * Update src/libraries/System.Private.CoreLib/src/System/String.Manipulation.cs * (MQ cleanup) Remove size_t from managed Brotli code (#44043) * Remove size_t from managed Brotli code * Apply suggestions from code review Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Stephen Toub <[email protected]> * Remove some unused icalls. (#44128) * Delete dead AppendSpanFormattable method from ValueStringBuilder (#44135) * Use slightly smaller code for String.Concat(object, object, ...) (#44133) C#, IL, and asm are all a bit smaller. We don't need to set the arguments to Empty if they're null, as the called function checks for null. * Fix equality check for unset variables (#44099) * Set GC_SUSPEND based on the target os not the host os. (#44093) * Set GC_SUSPEND based on the target os not the host os. * Add ENABLE_LLVM_RUNTIME cmake option. * Enable using the Ninja generator instead of Visual Studio on Windows builds. (#41897) * Move DacTableGen out of the CMake build and into tools-local. * Attempt: usenmakefiles means ninja. * Make changes to Windows build to support non-VS generators like ninja. * Use CMake 3.16 PCHs. * Remove explicit process count. * Build and run DacTableGen via the .NET CLI we pull down for the build instead of via CMake. * Update configurecompiler.cmake to specify language (needed for Ninja). Signed-off-by: Jeremy Koritzinsky <[email protected]> * Get Ninja build working again. * Pass configuration to cmake configure step for Ninja build. * Rename flag * Fix configure step for cross-arch native build Signed-off-by: Jeremy Koritzinsky <[email protected]> * Support building via Ninja for coreclr via an MSBuild property Signed-off-by: Jeremy Koritzinsky <[email protected]> * Implement support for opening the CoreCLR solution in VS via the -vs flag. * Fix setting generator to Ninja. * Merge libraries gen-buildsys-win.bat with runtime gen-buildsys.cmd to share windows cmake configure infra. * Enable building libraries native build with ninja. * Centralize setting CMAKE_SYSTEM_VERSION for windows builds. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Update host build to use shared gen-buildsys.cmd * Fix multi-config generator detection. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Exit with success when we build successfully. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Enable and fix up Ninja build for hosts. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Only link delayimp.lib on Windows Signed-off-by: Jeremy Koritzinsky <[email protected]> * Remove "overriding /W3 with /W1" warning in Ninja build of hosts by turning on /W3 and disabling any new warnings we hit. * Reduce command line override warnings by adopting the MSVC_RUNTIME_LIBRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows). * Use the delayed expansion syntax to correctly pass in the config to cmake. * Move all usages of MSVC runtime library selection to the CMake abstraction to reduce console spew about overridden flags. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Condition the rest of the compiler options correctly so they don't show up as invalid parameter warnings for assembly builds. * Correctly build DIALib and DacTableGen as AnyCPU * Fix WRITE_BARRIER_CHECK define Signed-off-by: Jeremy Koritzinsky <[email protected]> * Fix passing arch for MSBuild build. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Configure ASM compiler command line format in configurecompiler.cmake and use CMake's assembly compilation step to build ASM files for ARM/ARM64 instead of manually building them and adding the generated files as additional sources. * Cleanup after fixing ARM/ARM64 support. * Rename variable * Condition PCH's for C++-only as needed to fix using them on non-Windows (where we compile C as C instead of as C++). * Fix CoreCLR native test build on windows to point to new gen-buildsys location. * Add missing compilation options in Ninja build that are implicitly defined in the MSBuild build. * Disable analyzers for DacTableGen. Remove /MAP linker flag (that's for local testing). * Upgrade to CMake 3.16.4 * Don't warn on unused variables in CMake (these usually vary on platform/config) Signed-off-by: Jeremy Koritzinsky <[email protected]> * Manually compile ARM and ARM64 asm files when using the VS generator since it doesn't support emitting MARMASM item types in projects (and as a result cannot compile ARM or ARM64 asm without assistance). * Enable CMake policy in test build to use CMAKE_MSVC_RUNTIME_LIBRARY variable. * Update initially disabled warnings in host. * Add -ninja arg to root scripts to enable using Ninja. * Enable using Ninja in CoreCLR CI. * Try to fix passing the -ninja arg in yaml. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Add ninja to the Windows invocation * Fix 2-phase cross targeting to do one phase at a time and initialize the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets. * Restore old /Ox optimization setting for Release. * Fix typos, don't pass exception flags to C files. * Remove unneeded pragma. * Fix .pgd file installation. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Add mention of the `-ninja` flag in the Building CoreCLR documentation. Signed-off-by: Jeremy Koritzinsky <[email protected]> * Mention installation locations for Ninja and warn while building if CMake is older than 3.16.0. * Update docs/workflow/requirements/windows-requirements.md * Add issues.targets CG2 entries for issue #44054 (#44142) * Delete duplicate Buffer.Memmove implementations and avoid unnecessary pinning (#44118) * Delete duplicate Buffer.Memmove implementations and avoid unnecessary pinning We have several versions of Buffer.Memmove, including one implemented around `byte*` and one implemented around `ref byte`. We can delete the former and just use the latter everywhere. In doing so, we can also remove a few wrapper helpers, and reduce pinning in places where those wrappers were pinning to get pointers only to eventually end up in the ref-based implementation, anyway. * Use ThrowHelper in more places * Fix incorrect handling of character range and capitalization in regex (#42282) * Bug fix and unit test * Address comments * An elegant fix and unit tests * Remove old code * Fix the unit tests * sq * sq * Address comments and add test case * Skip the unit tests on net framework * Debug commit for CI * Try to get more error info * Run validation as a unit test. Skip it on non-ICU environments * Add NativeMainSource alternative to Android Sample App (#44076) * Add NativeMainSource alternative to Android Sample * Move nativeMainSource from parameter to property Co-authored-by: Mitchell Hwang <[email protected]> * Set the MSVC RuntimeLibrary properties for ASM_MASM when we use MASM. (#44155) * Remove OSX_arm64 from platform group all (#44160) * Update Brotli to v1.0.9 (#44107) * sync brotli source * update runtime project * Slight cleanup in hostpolicy (#44157) * Strip release builds on wasm. Avoid linking against libicudata.a, its not needed. (#43951) Fix warnings. * Add property to reuse Android App signing key (#44085) Co-authored-by: Mitchell Hwang <[email protected]> * Remove stale !s and TODO-NULLABLEs, plus those for indexer nullability tracking (#44136) * JSON continuation tests (#42393) * Repro #42070 * formatting * namespace * Fix * never forget the header * More tests - Test continuation at every position inside the tested object - Many member with primitive and nullable types - One more level of nested object - All combinations of class/struct for tested and nested object - tested and nested object with parametrized ctor for some properties * Addressed feedback * refactoring * Test with original repro data from #42070 * custom converter to ensure the padding is written in front of the tested object * rename * test data moved to Strings.resx * Using test data from SR * Generalize continuation tests for payloads of any length Tweak the payload and expect `JsonException` * merge * Test deserialize with Utf8JsonReader and ReadOnlySequence * Again with value typed nested object * Add tests for splitted whitespaces * Addressed feedback Added dictionary test * Validate line and position of failure in tweaked payloads more tweaks * Fixed comment Co-authored-by: Layomi Akinrinade <[email protected]> * Changing Windows_NT -> Windows for enabling TargetPlatformMoniker feature of the sdk for platform specific tfms. (#43651) * eng change * coreclr change * libraries change * tests change * installer & mono change * py files changed * .md change * .yml change * remove redundant targetos property * Windows_NT -> windows after ninja change * fixing runtime build failure * [interp] Fix pinvoke passing of valuetype on wasm (#44129) For valuetypes, a stackval used to contain a pointer to the valuetype data, while currently the valuetype data is stored directly in the stackval. * Move the exclusion of unregister01/03 into general CoreCLR group (#44163) * Remove some unsafe code from System.Xml (#43379) * Remove private DoubleToInt64Bits * Remove unsafe IntToString implementation * Remove unsafe AdjustLineInfo implementation * Remove unsafe GetStringAligned implementation * Remove unsafe code from BinHexDecoder * Remove unsafe MemoryMarshal usage from CodeIdentifier * Remove unsafe Base64Decoder implementation * Remove unsafe code in GetSingle and GetDouble * Remove unsafe code from CheckText * Fix an assert which is hit when compiling Microsoft.CodeAnalysis.dll with llvmonly. (#44092) * Fix asserts that were always true due to a missed neg. (#44095) * Fix asserts that were always true due to missed neg. * Ensure we don't assert for HWIntrinsics that already had the imm operand marked contained Co-authored-by: Tanner Gooding <[email protected]> * Use actual nint/nuint in Latin1Utility (#44162) * Add stream conformance tests for standalone streams (#44069) * Nullable-enable stream conformance tests * Add standalone stream conformance tests * Address PR feedback * Move serializable types for xml serialization generator tests to separate assembly (#44164) * Move serializable types for xml serialization generator tests to separate assembly * Add comment explaining why we need a separate assembly * Fix race condition in timeout handling and GetAsync_ContentCanBeCanceled (#44169) There are three issues here. Two tests issues: 1. When the client times out, it could do so while the server is still reading the request or sending the response, in which case the server can fail and cause the test to fail. The server failure needs to be ignored. 2. when the client times out, it could do so before it even initiates the request, in which case the test would hang while the server waits indefinitely for a request that'll never arrive. The server waiting needs to factor in the client's completion. And one product issue: 1. The timeout handling we added in .NET 5 has a race condition that can, in extreme conditions (e.g. a unit test trying to force the interleaving) result in us not throwing the appropriate TimeoutException. There are two different timer mechanisms used, the one based on Timer inside of CancellationTokenSource, and then the use of Envrionment.TickCount64 to track how much time has progressed. Their quantums are different, so it's possible for the timer to fire due to the timeout expiring but Environment.TickCount64 not ticking over, at which point we'll read it and determine that the cause of the cancellation wasn't for a timeout. The fix is to use the pending requests token source to check whether it or the timeout occurred, rather than using the time elapsed to determine which of the two occurred. * Fill out hardware intrinsics linker substitutions for unsupported platforms (#44156) Fix #44146 * Fix/remove TODO-NULLABLE in System.Private.Xml (#44149) * Fix/remove TODO-NULLABLE in System.Private.Xml * update contract for System.Xml.ReaderWriter * follow up changes * disable MockQuicStreamConformanceTests.Parallel_ReadWriteMultipleStreamsConcurrently (#44184) * disable MockQuicStreamConformanceTests.Parallel_ReadWriteMultipleStreamsConcurrently * Update QuicStreamConnectedStreamConformanceTests.cs Co-authored-by: Stephen Toub <[email protected]> * Fix comment causing formatting leg in CI to fail (#44197) * Add SYSLIB0013 into obsoletion list (#44203) * Add SYSLIB0013 into obsoletion list Add obsoletion of Uri.EscapeUriString to the list. * Remove extra new line * Clean up style of Random.cs (#44195) No functional changes. * Use different variable in base-job.yml to avoid confusion (#44193) `/p:RuntimeOS` and the `-os` args are not the same thing (see https://github.com/dotnet/runtime/issues/33419) so we shouldn't be using the same variable for them. * [mono] remove 2 extra members from System.Object (#44081) 1. move `GetRawData` to an extension method in RuntimeHelpers and fixup the intrinsics 2. replace `Object.CloneInternal` by `MarshalAsAttribute.CloneInternal` at its one use site 3. re-enable System.Reflection.Tests.TypeInfoTests.FindMembers and fix mono/mono#15029 * TcpClient.ConnectAsync(EndPoint) (#44110) Implement TcpClient.ConnectAsync(IPEndPoint) * Extend SendAsync/ReceiveAsync cancellation tests (#44161) - Rename CanceledDuringOperation_Throws to ReceiveAsync_CanceledDuringOperation_Throws and extend it to IPv6 sockets. - Introduce cancellation test for Socket.SendAsync. * improve accuracy of Expect100Continue_WaitsExpectedPeriodOfTimeBeforeSendingContent test (#44053) * improve accuracy of Expect100Continue_WaitsExpectedPeriodOfTimeBeforeSendingContent test * feedback from review * Don't create a wrapper for run-v8.sh (#44172) * Check catch and throw non-Exception derived types (#43969) Verifies: * `X` is derived from `Exception` in `throw X`. * `throw null` is valid. * `X` is derived from `Exception` in `catch(X)`. Contributes to #37390 * Don't expect LIST nodes in CALL args. (#44089) * Remove several volatiles from Timer (#44199) TimerQueueTimer._canceled is only ever used when _associatedTimerQueue is held. TimerQueueTimer._notifyWhenNoCallbacksRunning only ever transitions from null to non-null, always under a lock, and the only time it's used outside of a lock is just after that same thread validated the field was non-null while holding the lock. * Try a different attribute for disabling stream conformance tests on browser (#44218) * Check that we don't create null checks on an address that is not null. (#44059) * Check that we don't create null checks on an address that is not null. * Egor's request * [interp] Reinitialize error in a few places (#44188) Otherwise we can assert when attempting to use the error. * Remove the old mono timezone icall implementation from driver.c (#44171) * Remove unused on netcore icall and set TZ in the startup js * Add a PNSE configuration for System.Net.Quic (#44223) Systme.Net.Quic is part of the shared framework but dosn't have a configuration that applies to Browser. Adding a non-rid configuration so that this applies for all future configurations that aren't explicitely listed as well. Changing some throw null lambdas in the reference source file to work around a bug in the GenFacades not supported assembly creation logic: https://github.com/dotnet/arcade/issues/6509. * Add wiggle room to MonitoringIsEnabled test (#44228) * Add assert to SharedPerformanceCounter to debug test failures in CI (#44225) * Tolerate missing tests (#44168) * Remove trailing whitespace * Tolerate missing test file when analyzing results Before, we'd hit an assert. * Wasm async entrypoint (#44045) * [browser][bindings] Add support for calling the async Task entry point. - When the entry point of an assembly is async we need to execute that method and return the Task object - The Task object that is return from the execution of the async entry point will be marshaled to a JavaScript Promise. Co-authored-by: Kenneth Pouncey <[email protected]> * [interp] Simplify GetRawData (#44217) Generate also better code. * Doubly linked freelist fixes (#43636) * Fix bugs in the doubly linked free list logic and enable it. Details: - #define DOUBLY_LINKED_FL to enable - Fix issue in adjust_limit with accidentally setting MAKE_FREE_OBJ_IN_COMPACT bit in the plug_and_gap structure that gets put before a pinned plug - this overwrites any object that used to be there (after it has been saved elsewhere), so added logic to detect the situation and set MAKE_FREE_OBJ_IN_COMPACT in the saved object instead. To make this work, added new field "saved_pinned_plug_index" to remember which pinned plug to look at. - In connection with the previous issue in adjust_limit, fix case of a 3 pointer size plug in from of a pinned plug - for this we need a range test to catch the case, plus add this offset to the beginning of saved_pre_plug_reloc. On the other hand, 3 pointers is the minimum size object, so added an assert to that effect as suggested in code review. - Changed several comparisons because an object <= min_free_item_no_prev cannot be on the free list. - Fixed issue in should_set_bgc_mark_bit where an assert fired because current_sweep_seg was null because the background gc thread was about to sweep, but hadn't set current_sweep_seg just yet. Fix is to set current_sweep_seg and current_sweep_pos early. * Add 'condition: always()' to Disk Usage after Build (#44209) Looking at builds like this: https://dev.azure.com/dnceng/public/_build/results?buildId=873478&view=logs&j=1f8f000c-1adc-5434-677a-95dd9e006aba&t=30ef7da5-6e59-540d-0dbe-fa78537a8165&s=d654deb9-056d-50a2-1717-90c08683d50a, the other legs are failing left and right filling the agents' disks. We need to know the disk usage after a build to see if this is relevant to build failures. * Socket.Tests: remove TestSettings.UDPRedundancy (#44202) dotnet/corefx#15697 changed this value to 1, and no-one touched it since then. As the comment in that PR points out, the root cause of the UDP test failures wasn't packet loss, but IPv4 / IPv6 port collision on Unix in dual-mode cases. The value and the for loops are complicating Socket test code unnecessarily, simplification seems reasonable. * Fix: NetworkStream throwing inconsistent exceptions (#40772) Fix a bug: Span<byte> overloads of NetworkStream throwing ObjectDisposedException instead of NetworkException, when not using derived NetworkStream. * Reenable dllmap for iOS/Android (#44243) This got lost as part of the CMake conversion but we still need it for these platforms. Fixes https://github.com/dotnet/runtime/issues/44242 * Add stream conformance tests for SocketsHttpHandler HTTP/1.1 streams (#44204) * Add stream conformance tests for SocketsHttpHandler HTTP/1.1 streams * Address PR feedback * Respect DOTNET_SHUTDOWNTIMEOUTSECONDS in HostOptions (#44080) * Respect DOTNET_SHUTDOWNTIMEOUTSECONDS in HostOptions * Add Initialize to HostOptions, add HostOptions configure * Add test * Fix inject_debug_resources target (#44238) * Update buildtriage.md (#44196) * Update buildtriage.md * Add debugging a ci dump documentation template and script to fill info and include it on azdo test tab (#43937) * Add debugging a ci dump documentation template and script to generate it on helix * PR Feedback and fix helix workitems * Fix python script and -buildid argument * Upload doc only if dumps were found * Pass down templatedir * PR feedback * Use SetFileInformationByHandle on FileStream.SetLength (#44170) * Use SetFileInformationByHandle on FileStream.SetLength * Remove Interop.SetEndOfFile since is no longer used * Add manual test * Address feedback * JIT: simple redundant compare optimization (#43811) For a relop, walk up the dominator tree to see if any dominating block has a similar compare. If so, and there's just one path from that block to the relop, the relop's value is known. Closes #11909. * Remove MNADeprecationWorkaround in host test assets (#44259) * Update IpcStreamFactory state machine to handle being started on a thread that ends (#43711) Handles the case on Windows where Async IO is cancelled due to the thread coreclr started on ending. * Use shared lock for files opened without FileAccess.Write on Linux (#44185) * Use shared lock for readonly files on Linux * Add suggested tests * Exclude test from OSX * Add RemoteExecutor.IsSupported condition to tests * Disable DOUBLY_LINKED_FL (#44262) Fixes #44250 * Move dotnet version to hardcoded version to workaround TFM change (#44224) * Move dotnet version to hardcoded version to workaround TFM change * Change 'windows' -> 'Windows_NT' * Move osGroup checks back to 'windows' * Update scenario yml * Add crossgen path for SoD tests * Split SslStream stream conformance tests for TLS1.2/1.3 (#44245) * Allow embedded nulls in the result of mono_wasm_invoke_js (). (#44237) Fixes https://github.com/dotnet/runtime/issues/42693. * Omit ErrorInformation from ManagementException binary serialization (#44112) * Omit ErrorInformation from ManagementException binary serialization Fixes #2116 * Serialize MgmtException.errorObject as NULL for .NET Framework compat * Add R2RDump support for lazy GC info (#44261) This change makes it such that GC info is not automatically parsed for each method, we only store a delegate performing the parsing that gets called on demand. In practice this also helps us with working around a bug in GC info previously crashing R2RDump when emitting PDB in ARM release build mode. Thanks Tomas * More consistent cache key validation in Remove (#44286) * fix stream conformance tests for mock provider (#44282) Co-authored-by: Geoffrey Kizer <[email protected]> * Correctly set dylib version for libmonosgen-2.0.dylib (#44290) Fixes https://github.com/dotnet/runtime/issues/44284 * Uncomment `pr: none` in perf.yml (#44302) * Wor…

imhameed mentioned this pull request Oct 22, 2020

CoreCLR runtime tests + Mono LLVM AOT on arm64 Linux dotnet/runtime#41751

Merged

akoeplinger requested a review from mthalman October 22, 2020 19:00

mthalman approved these changes Oct 22, 2020

View reviewed changes

mthalman merged commit 219667d into dotnet:master Oct 22, 2020

imhameed added a commit to imhameed/runtime that referenced this pull request Oct 23, 2020

Use an Ubuntu 16.04 cross compilation image with clang-tools-9 preins…

45896f1

…talled. See also: dotnet/dotnet-buildtools-prereqs-docker#375

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install clang-tools-9 on Ubuntu 16.04 and 18.04 cross-building images. #375

Install clang-tools-9 on Ubuntu 16.04 and 18.04 cross-building images. #375

imhameed commented Oct 22, 2020

Install clang-tools-9 on Ubuntu 16.04 and 18.04 cross-building images. #375

Install clang-tools-9 on Ubuntu 16.04 and 18.04 cross-building images. #375

Conversation

imhameed commented Oct 22, 2020