Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge from dotnet/runtime #1761

Merged

Conversation

MichalStrehovsky
Copy link
Member

No description provided.

thaystg and others added 30 commits November 30, 2021 17:02
* [Issue: 61620] Optimizing ARM64 for *x = 0;

* Update src/coreclr/jit/lower.cpp

Co-authored-by: SingleAccretion <[email protected]>

* Fixed bug with * x = dConst if dConst is not 0

* remove extra printf

* Replacing IsFPZero with IsCnsNonZeroFltOrDbl for STOREIND Minor edits with conditional compilation in lower.cpp

* fixed ARM codegen for STOREIND

* Update src/coreclr/jit/lower.cpp

Co-authored-by: SingleAccretion <[email protected]>

* Update src/coreclr/jit/lower.cpp

Co-authored-by: SingleAccretion <[email protected]>

* fix formatting

Co-authored-by: SingleAccretion <[email protected]>
… that we use the RTM sdk. (#62156)

Co-authored-by: Andy Gocke <[email protected]>
…106)

* Always use int3/bkpt

* Use right formatting for brk instruction

* Rename to brk_unix and brk_windows

* jit format
* Fix a few syntax issues in shell scripts

* Cleanup trailing whitespace in changed files
```sh
# git remote add dotnet https://github.com/dotnet/runtime && git pull --rebase dotnet main

if uname 2>/devnull | grep -q Darwin; then
    space=" "
fi

git show --name-only --pretty="" HEAD...dotnet/main |\
    xargs -I{} sh -c "test -f {} && sed -i$space'' 's/[[:space:]]*$//' {}"
```

* Address CR feedback
* Improve SuperPMI temp filename creation

1. Disallow all non-ASCII characters from the MC filename.
2. Disallow `(` and `)` in filenames.
3. To improve uniqueness, always append an 8-character random number
to the filename.

* Fix Linux build
* Add unmark loopAlign in a missing place

* Add test case

* add trailing \n
System.Linq.Expressions currently offers multiple build time definitions to build different flavors of the library:

* `FEATURE_COMPILE` (defined everywhere except iOS/tvOS/Catalyst, and NativeAOT) - enables Reflection.Emit-based expression tree compiler.
* `FEATURE_INTERPRET` (always defined and not actually possible to build without) - enables the expression interpreter
* `FEATURE_DLG_INVOKE` (defined everywhere except NativeAOT, but we likely need to be able to run without it on iOS too because there's uninvestigated bugs around it ActiveIssue'd in #54970) - in the interpreter, use a delegate to invoke `CallInstructions` instead of `MethodInfo.Invoke`. The delegate might have to be Reflection.Emitted if it's not supportable with `Func`/`Action` (that's the uninvestigated iOS/tvOS/Catalyst bug).

For #61231, we need to be able to build a single System.Linq.Expression library that can switch between these build-time configurations _at publish time_ since we don't want to build a separate S.L.Expressions library for NativeAOT. There are advantages in having this setup for non-NativeAOT scenarios too. This pull request accomplishes that by mechanically changing the `#define`s into feature switches.

The feature switch is placed in the last proposed location for dotnet/runtime#17973. I expect we'll want such API to be public at some point; now that Xamarin and NativeAOT use this formerly .NET Native-only thing the API request became relevant again.

This pull request is focused on the mechanical replacement of `#defines` with feature switches and it's already a lot bigger than I'm comfortable with.

There's some obvious "`!FEATURE_COMPILE` means this is .NET Native with everything what that meant" that I did not touch because this is meant to be a mechanical change. Some cleanup will be needed at some point. Right now this just mostly means we're running fewer tests than we could.

Validation:
* Verified that we're still running the same number of tests with CoreCLR as we previously were and they're all passing.
* Verified we're getting mostly the same size of the S.L.Expressions library on iOS (433 kB grew to 436 kB, the diffs are expected).
* Verified things work on the NativeAOT side as well.
* don't run the System.IO.FileSystem.DriveInfo in parallel, fixes #62119

* unify disabling test parallelization per assembly
…ds. (#62153)

* Console.Unix: fix OpenStandardInput Stream sometimes throwing for Reads.

When connected to a terminal reads happen line-by-line.

StdInReader caches the line in a StringBuilder.

Instead of returning when the destination buffer is full, an attempt
was made to decode the next chunk from the StringBuilder (if there is
one). This causes the encoder to throw for encoding into an empty buffer.
* Propagate debug info in loop cloning

* Do not consume debug info on the standalone call statement created for
  inline candidates. This would lead to the loss of debugging
  information for failed inline candidates, since those statements are
  dropped and expanded in the upcoming GT_RET_EXPR node instead. In some
  cases it would also lead to the loss of debugging information for
  successful inlines.
  In the new logic we allow the same debugging information to be
  attached to the upcoming statement using the GT_RET_EXPR.

This change adds around 40 KB (~0.5%) to SPC.
Fixes issue observed in dotnet#1128 (comment) - the trees generate additions of `TYP_I_IMPL` with `TYP_INT` and that confuses things down the line.
Use TYP_UNDEF (as TYP_MEM) for "first field"-like maps and
TYP_UNKNOWN (as TYP_HEAP) for the heaps themselves.

This will enable the type validation logic to be more precise.
For now, just assert that the heap is always updated with the
properly typed VNs.
- Convert RuntimeHelpers.InitializeArray to new style intrinsic
- Delete unused isStructRequiringStackAllocRetBuf JIT/EE method and related optimization
- Delete unused CORINFO_HELP_GETSTATICFIELDADDR_CONTEXT jit helper
* Optimize FMA codegen base on the overwritten

* Improve function/var names

* Add assertions

* Get use of FMA with TryGetUse

* Decide FMA form with two conditions, OverwrittenOpNum and isContained

* Fix op reg error in codegen

* Decide form using lastUse and isContained in no overwritten case

* Clean up code

* Separate default case overwrittenOpNum==0

* Apply format patch

* Change variable and function names

* Update regOptional for op1 and resolve some other comments

* Optimize FMA codegen base on the overwritten

* Improve function/var names

* Add assertions

* Get use of FMA with TryGetUse

* Decide FMA form with two conditions, OverwrittenOpNum and isContained

* Fix op reg error in codegen

* Decide form using lastUse and isContained in no overwritten case

* Clean up code

* Separate default case overwrittenOpNum==0

* Apply format patch

* Change variable and function names

* Update regOptional for op1 and resolve some other comments

* Change var names

* Fix jit format

* Fix build node error for op1 is regOptional

* Use targetReg instead of GetResultOpNumForFMA in codegen

* Update variable names

* Refactor lsra to solve lastUse status changed caused assertion failure

* Add check to prioritize contained op in lsra

* Update for jit format

* Simplify code

* Resolve comments

* Comment out assert because of lastUse change

* Fix some copiesUpperBits related errors

* Update src/coreclr/jit/lsraxarch.cpp

Co-authored-by: Kunal Pathak <[email protected]>

* Add link to the new issue

Co-authored-by: Kunal Pathak <[email protected]>
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
It looks like in CI we are seeing a null method ID in some cases when
this test is run under R2R and GC stress. Add a check for this, and also
mark the test as stress incompatible like other ETW tests, since it runs
very slowly under stress modes.

Fix #62118
* Added parameter for processing the value of the configuration e.g. for obfuscation

* Rename

* Extension methods separated

* After review suggestion changes

* Documentation update

* Build fixes

* Documentation for ConfigurationDebugViewContext added

* Nullable string error fix

* Changed to nullable string

* PR fixes

* Definition fix

* Missing definition

* Added getters to the definition

* Update src/libraries/Microsoft.Extensions.Configuration.Abstractions/src/ConfigurationRootExtensions.cs

Co-authored-by: Santiago Fernandez Madero <[email protected]>

* Tests added

* Update src/libraries/Microsoft.Extensions.Configuration.Abstractions/src/ConfigurationDebugViewContext.cs

Docs update

Co-authored-by: Eric Erhardt <[email protected]>

* Update src/libraries/Microsoft.Extensions.Configuration.Abstractions/src/ConfigurationDebugViewContext.cs

Docs updateDocs update

Co-authored-by: Eric Erhardt <[email protected]>

* Update src/libraries/Microsoft.Extensions.Configuration.Abstractions/src/ConfigurationDebugViewContext.cs

Docs update

Co-authored-by: Eric Erhardt <[email protected]>

* Update src/libraries/Microsoft.Extensions.Configuration.Abstractions/src/ConfigurationDebugViewContext.cs

Docs update

Co-authored-by: Eric Erhardt <[email protected]>

* Update src/libraries/Microsoft.Extensions.Configuration.Abstractions/src/ConfigurationRootExtensions.cs

Docs update

Co-authored-by: Eric Erhardt <[email protected]>

* Update src/libraries/Microsoft.Extensions.Configuration.Abstractions/src/ConfigurationDebugViewContext.cs

Docs update

Co-authored-by: Eric Erhardt <[email protected]>

Co-authored-by: Santiago Fernandez Madero <[email protected]>
Co-authored-by: Eric Erhardt <[email protected]>
* Update dependencies from https://github.com/dotnet/linker build 20211115.4

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21565.4

* Update dependencies from https://github.com/dotnet/linker build 20211116.4

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21566.4

* Update dependencies from https://github.com/dotnet/linker build 20211117.1

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21567.1

* Update dependencies from https://github.com/dotnet/linker build 20211118.1

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21568.1

* Update dependencies from https://github.com/dotnet/linker build 20211119.1

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21569.1

* Update dependencies from https://github.com/dotnet/linker build 20211122.2

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21572.2

* Update dependencies from https://github.com/dotnet/linker build 20211130.1

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21580.1

* Update dependencies from https://github.com/dotnet/linker build 20211201.1

Microsoft.NET.ILLink.Tasks
 From Version 7.0.100-1.21562.1 -> To Version 7.0.100-1.21601.1

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Alexander Köplinger <[email protected]>
…ze it away causing an assert inside 'opt'. (#62198)
* Added a new optional -compile argument that specifies which method contexts need to be compiled

* Update src/coreclr/scripts/superpmi.py

Co-authored-by: Bruce Forstall <[email protected]>

* Added another form of -compile argument like -c

Co-authored-by: Bruce Forstall <[email protected]>
…pArchive (#60973)

* throw exception when creating an entry name that already exists in ZipArchive

* secure code at the top of CreateEntry call

* specify entry name in exception message

* fix exception message, make AddEntry retrocompatible, and fix test issue with globalization

* restore the EmptyEntryTest code as before
@MichalStrehovsky MichalStrehovsky added the area-NativeAOT-coreclr .NET runtime optimized for ahead of time compilation label Dec 2, 2021
@MichalStrehovsky
Copy link
Member Author

Oh good, DllImportGenerator binplaces itself somewhere where we consider it part of the framework.

Maybe we can gracefully handle this one.

[D:\a\_work\1\s\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\nativeaot\build\BuildFrameworkNativeObjects.proj]
2021-12-02T02:40:47.3941457Z   Internal.TypeSystem.TypeSystemException+FileNotFoundException: Failed to load assembly 'Microsoft.CodeAnalysis.Workspaces'
2021-12-02T02:40:47.4279226Z      at Internal.TypeSystem.ThrowHelper.ThrowFileNotFoundException(ExceptionStringID id, String fileName)
2021-12-02T02:40:47.4842620Z      at Internal.TypeSystem.ResolutionFailure.Throw()
2021-12-02T02:40:47.5516622Z      at Internal.TypeSystem.Ecma.EcmaModule.GetType(EntityHandle handle)
2021-12-02T02:40:47.6595806Z      at Internal.TypeSystem.Ecma.EcmaType.InitializeBaseType()
2021-12-02T02:40:47.7145432Z      at Internal.TypeSystem.TypeDesc.get_IsDelegate()
2021-12-02T02:40:47.8321293Z      at ILCompiler.DependencyAnalysis.TypeMetadataNode.GetStaticDependencies(NodeFactory factory)
2021-12-02T02:40:47.9451587Z      at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.GetStaticDependenciesImpl(DependencyNodeCore`1 node)
2021-12-02T02:40:48.0827007Z      at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.GetStaticDependencies(DependencyNodeCore`1 node)
2021-12-02T02:40:48.1301405Z      at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ProcessMarkStack()
2021-12-02T02:40:48.2247466Z      at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes()
2021-12-02T02:40:48.3161684Z      at ILCompiler.RyuJitCompilation.CompileInternal(String outputFile, ObjectDumper dumper)
2021-12-02T02:40:48.3941818Z      at ILCompiler.Compilation.ILCompiler.ICompilation.Compile(String outputFile, ObjectDumper dumper)
2021-12-02T02:40:48.4579501Z      at ILCompiler.Program.Run(String[] args)
2021-12-02T02:40:48.5047892Z      at ILCompiler.Program.Main(String[] args)
2021-12-02T02:40:48.5821786Z D:\a\_work\1\s\artifacts\tests\coreclr\windows.x64.Release\Tests\Core_Root\nativeaot\build\Microsoft.NETCore.Native.targets(293,5): error MSB3073: The command ""D:\a\_work\1\s\artifacts\tests\coreclr\windows.x64.Release\Tests/Core_Root/nativeaot\tools\ilc" @"D:\a\_work\1\s\artifacts\obj\/NativeAOTFX\Microsoft.Interop.DllImportGenerator.ilc.rsp"" exited with code 1.

@jkotas
Copy link
Member

jkotas commented Dec 2, 2021

Oh good, DllImportGenerator binplaces itself somewhere where we consider it part of the framework.

We are not hitting this problem for other source generators. What makes DllImportGenerator special?

Looks like that this was introduced by dotnet/runtime#62156. cc @jkoritzinsky

@jkoritzinsky
Copy link
Member

DllImportGenerator is the only generator that we reference within the shared framework other than the CoreLib-only generator (which gets bin placed differently).

@jkotas
Copy link
Member

jkotas commented Dec 2, 2021

I assume that there is filter somewhere that prevents the DllImportGenerator from being included in the shipping shared framework set. Is that right? So the problem is that this filter is not kicking for NativeAOT (or NativeAOT tests at least).

@jkotas
Copy link
Member

jkotas commented Dec 2, 2021

It sounds like that it is just a problem for tests. The DllImportGenerator is not in the .nupkg.

@jkoritzinsky
Copy link
Member

Yeah we don't include it in the shared framework today. That's a separate step that we still need to do when we get ready to ship it.

@jkotas
Copy link
Member

jkotas commented Dec 4, 2021

It sounds like that it is just a problem for tests

Looks similar to dotnet/runtime#62372

@MichalStrehovsky
Copy link
Member Author

Looks similar to dotnet/runtime#62372

Thanks, yes, looks similar. I've made a quick local workaround.

I also looked into fixing the compiler to just handle this input because it's just garbage input like any other, but this is the known issue with nested types - a nested type that is fine by itself is nested under something with a damaged base type. Because the type is nested, we need to put the owning type into the dependency graph, but having a type with a broken base hierarchy in the dependency graph requires adding a lot of try/catch in various places. We generally try to avoid broken things in the graph. The ultimate fix would be to rewrite metadata generation in the compiler to be S.R.Metadata-based, not type system based. Then these things are less annoying to deal with.

@MichalStrehovsky MichalStrehovsky merged commit 8277260 into dotnet:feature/NativeAOT Dec 7, 2021
@MichalStrehovsky MichalStrehovsky deleted the merge-main branch December 7, 2021 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-NativeAOT-coreclr .NET runtime optimized for ahead of time compilation
Projects
None yet
Development

Successfully merging this pull request may close these issues.