Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NativeAOT] Metadata not trimmed for nested Interface types when compiling TerraFx.Interop.Windows #66716

Closed
jkotas opened this issue Mar 16, 2022 · 11 comments · Fixed by #100000
Labels
area-NativeAOT-coreclr in-pr There is an active PR which will close this issue when it is merged
Milestone

Comments

@jkotas
Copy link
Member

jkotas commented Mar 16, 2022

Same repro as #66593 (comment)

Actual result: Nested Interface types like TerraFX.Interop.Windows.IEnumWbemClassObject+Interface types are present in the final binary

Expected result: TerraFX.Interop.Windows.IEnumWbemClassObject+Interface types are not present in the final binary (IL linker is able to trim these)

@jkotas
Copy link
Member Author

jkotas commented Mar 16, 2022

I have tried to debug this using dgml log by adding --fulllog --dgmllog c:\repro\log.dgml to the command line. The generation of dgmllog is crashing with: An item with the same key has already been added. Key: (ILCompiler.DependencyAnalysis.PrecomputedDictionaryLayoutNode, ILCompiler.DependencyAnalysis.NativeLayoutTemplateTypeLayoutVertexNode)

@jkotas
Copy link
Member Author

jkotas commented Mar 16, 2022

@MichalStrehovsky Does this ring any bells?

@agocke
Copy link
Member

agocke commented Mar 16, 2022

@jkotas Fixed by #66593?

@jkotas
Copy link
Member Author

jkotas commented Mar 16, 2022

Fixed by #66593?

No. This issue is what's left after fixing #66593.

@agocke agocke removed the untriaged New issue has not been triaged by the area owner label Mar 16, 2022
@agocke agocke added this to the 7.0.0 milestone Mar 16, 2022
@MichalStrehovsky
Copy link
Member

We don't trim interface lists. I played with it in the past (MichalStrehovsky/runtimelab@2a702bc), but it's a bit more involved (the above commit was enough to get a size estimate and I remember the size saving being interesting - more than 1% - but not interesting enough to spend time on it at that point).

It will require changing most of the occurrences of .RuntimeInterfaces in the dependency analysis within the compiler to use some kind of indirection. It will likely be something similar to how we track virtual method use.

I'll put this into Future since this is a nice to have size optimization.

@MichalStrehovsky MichalStrehovsky modified the milestones: 7.0.0, Future Mar 18, 2022
MichalStrehovsky added a commit to MichalStrehovsky/runtime that referenced this issue May 31, 2022
Fixes issue reported in dotnet#66716 (comment).

We create a hashset for the purpose of avoiding this problem, check it, but never add to it.
@MichalStrehovsky
Copy link
Member

I had a quick look - the only reason why we're generating the interfaces is because we're generating the struct that implements them. And the only reason why we generate the struct is because we called some instance methods on it and it became a potential target of reflection in the "IL Linker compat mode" we run by default (where whatever is called is considered reflection-accessible at runtime).

In the strict mode <IlcTrimMetadata>true</IlcTrimMetadata> we don't generate any of that. The repro in the top-post is 20% smaller with that setting.

I wish we just enabled the strict mode by default...

@jkotas
Copy link
Member Author

jkotas commented May 31, 2022

I wish we just enabled the strict mode by default...

I have been thinking about the recently. I think we should enable the strict mode by default.

jkotas pushed a commit that referenced this issue May 31, 2022
Fixes issue reported in #66716 (comment).

We create a hashset for the purpose of avoiding this problem, check it, but never add to it.
@jkotas
Copy link
Member Author

jkotas commented May 31, 2022

The default experience today is that you always see these messages:

  • Generating compatible native code. To optimize for size or speed, visit https://aka.ms/OptimizeNativeAOT
  • Optimizing assemblies for size, which may change the behavior of the app. Be sure to test after publishing.

We should change the default experience so that you only see Generating native code by default, enable IlcTrimMetadata by default, get all assemblies treated as trimmable by default (ie do not root app, etc.).

If there are no trimming or AOT warnings, it is all you get. Nice and clean experience.

If there are trimming or AOT warnings, we print additional message with a link to a top-level doc that talks about fixing the warnings properly and that has list of workarounds (e.g. setting IlcTrimMetadata to false, etc.).

Thoughts?

@jkotas
Copy link
Member Author

jkotas commented May 31, 2022

The conservative defaults made sense before we had the analyzers and enabling AOT was a guesswork. Now that we have the analyzers, we should be able to be more aggressive by default.

@MichalStrehovsky
Copy link
Member

The Optimizing assemblies for size, which may change the behavior of the app. Be sure to test after publishing. message shouldn't show up after dotnet/sdk#22636 - it will only show up if warnings are disabled.

I can definitely get behind enabling IlcTrimMetadata by default. The only observable difference with that setting will be around code that is already generating a trimming warning. (For anyone else reading this, IlcTrimMetadata skips generating reflection metadata for members that are not visible targets of reflection, kind of similar to dotnet/linker#1282, but with more stripping and less breaking of debugging experience since debug symbols are separate).

get all assemblies treated as trimmable by default (ie do not root app, etc.).

But this one I would like to do consistently - we either do this with IL Linker too, or we don't. It doesn't feel good for the AOT business to be more breaking than PublishTrimmed when it comes to trimming. We had a discussion about what the default should be in .NET 6. We went with rooting all assemblies that are not manifested as trimmable. There is an argument for doing the more aggressive thing - pretty much all our trimming warnings are not relevant for most apps because the target of their reflection is not in a trimmable assembly - so we spam dozens of warnings and the app still works fine. It feels better to generate fewer warnings that are all relevant (and the app being more likely to be broken) and have the old behavior behind another opt in.

Cc @dotnet/ilc-contrib for thoughts.

@jkotas
Copy link
Member Author

jkotas commented Jun 1, 2022

It feels better to generate fewer warnings that are all relevant (and the app being more likely to be broken)

Agree.

Also, I would like to keep emphasizing the importance of fixing the trimming warnings. The current defaults dilute this message since the conservative default covers up a lot of the issues, but not all of them.

MichalStrehovsky added a commit to MichalStrehovsky/runtime that referenced this issue Dec 6, 2023
The compiler can generate two kinds of `MethodTable` structures: constructed and unconstructed. The constructed one has a fully populated vtable and GCInfo and is required whenever the object type could be allocated on the heap. The unconstructed one is generated for all other scenarios as a size optimization.

We were previously also skipping emission of the interface list in the unconstructed case. But interface list might be required for variant casting. We could introduce yet another `MethodTable` kind for this specific scenario, but it doesn't seem to warrant the complexity.

Emitting interface list for all types is a less than 0.1% size regression for the Todos app. It is a 0.5% size regression for Hello World. That part is unfortunate. It's mostly due to the useless numeric interfaces. We can get this size back if we do dotnet#66716 and start trimming interface lists.

Fixes dotnet#95574.
jkotas pushed a commit that referenced this issue Dec 8, 2023
* Generate interface lists for necessary EETypes

The compiler can generate two kinds of `MethodTable` structures: constructed and unconstructed. The constructed one has a fully populated vtable and GCInfo and is required whenever the object type could be allocated on the heap. The unconstructed one is generated for all other scenarios as a size optimization.

We were previously also skipping emission of the interface list in the unconstructed case. But interface list might be required for variant casting. We could introduce yet another `MethodTable` kind for this specific scenario, but it doesn't seem to warrant the complexity.

Emitting interface list for all types is a less than 0.1% size regression for the Todos app. It is a 0.5% size regression for Hello World. That part is unfortunate. It's mostly due to the useless numeric interfaces. We can get this size back if we do #66716 and start trimming interface lists.

Fixes #95574.

* Update NecessaryCanonicalEETypeNode.cs
MichalStrehovsky added a commit to MichalStrehovsky/runtime that referenced this issue Mar 20, 2024
So far, the compiler could only trim unused interface __methods__; the interface __types__ themselves were left alone. There is not a huge cost associated with having these extra `MethodTable`s, but it sill impacts size/working set/startup. Initial estimate showed this could actually be up to 2% for BasicMinimalApi so I decided to investigate a fix.

This is an attempt to start trimming them. I chose a relatively conservative approach:

* Stop unconditionally rooting the interface `MethodTable` in the implementing class `MethodTable` InterfaceList. Instead check whether someone else marked it first.
* Track whether an interface type definition is used in any way. This avoids problem with variance, etc. If an interface definition is used, all instantiations that we were trying to optimize away get the `MethodTable` and won't be optimized. We should be able to do better than this, maybe, at some point.
* Classes that implement generic interfaces with default methods need special treatment because we index into interface list at runtime and we might not know the correct index yet. So just forget the optimization in that case.

Fixes dotnet#66716.
@dotnet-policy-service dotnet-policy-service bot added the in-pr There is an active PR which will close this issue when it is merged label Mar 20, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Apr 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-NativeAOT-coreclr in-pr There is an active PR which will close this issue when it is merged
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants