Optimizing generics by using ILCompiler dependency graph and profile guided optimization in Mono AOT

[kotlarmilos]

Description

Use ILCompiler (NativeAOT compiler) to perform the “pre-pass” analysis and instruct Mono AOT compiler about all the referenced generic types in the program which are only statically reachable. The exchanged information could a list of inflated methods of a generic type for which it can be proven that is not accessed dynamically, in MIBC format as an input for Mono AOT compiler.

To determine which specific instances of generic methods are to be used during runtime and provide this information to the Mono AOT compiler, profile guided optimization (PGO) approach could be used.

This would remove the need for generating generic sharing methods and additional specific instantiations for generic methods and reduce application’s footprint. It should be possible to reuse some of the managed code comprising the dotnet-pgo tool. It would be good to pull the shared code out into a separate assembly (or continue with the current approach that ilc/dotnet-pgo use - share source files).

Tasks

• ILCompiler dumps dependency graph in a .mibc format
• Enable PGO in Mono AOT engine full-LLVM mode
• Integration and testing of the ILCompiler's dependency graph and the PGO
• PGO with ILCompiler in Mono AOT available as an experimental feature

We still have open questions related to the integration and what is required to introduce it as an experimental feature in .NET 8, so comments and feedback are welcome.

/cc: @lambdageek @SamMonoRT

[ghost]

Tagging subscribers to 'os-ios': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Use ILCompiler (NativeAOT compiler) to perform the “pre-pass” analysis and instruct Mono AOT compiler about all the referenced generic types in the program which are only statically reachable. The exchanged information could a list of inflated methods of a generic type for which it can be proven that is not accessed dynamically, in MIBC format as an input for Mono AOT compiler.

To determine which specific instances of generic methods are to be used during runtime and provide this information to the Mono AOT compiler, profile guided optimization (PGO) approach could be used.

This would remove the need for generating generic sharing methods and additional specific instantiations for generic methods and reduce application’s footprint. It should be possible to reuse some of the managed code comprising the dotnet-pgo tool. It would be good to pull the shared code out into a separate assembly (or continue with the current approach that ilc/dotnet-pgo use - share source files).

Tasks

• ILCompiler dumps dependency graph in a .mibc format
• Enable PGO in Mono AOT engine full-LLVM mode
• Integration and testing of the ILCompiler's dependency graph and the PGO
• PGO with ILCompiler in Mono AOT available as an experimental feature

We still have open questions related to the integration and what is required to introduce it as an experimental feature in .NET 8, so comments and feedback are welcome.

/cc: @lambdageek @SamMonoRT

Author:	kotlarmilos
Assignees:	kotlarmilos, ivanpovazan, LeVladIonescu
Labels:	area-Codegen-AOT-mono, feature-request, os-ios
Milestone:	8.0.0

[SamMonoRT]

/cc: @lateralusX

[ivanpovazan]

[ ] ILCompiler dumps dependency graph in a .mibc format

Here you can find progress regarding the task: main...ivanpovazan:runtime:nativeaot-mono

The https://github.com/ivanpovazan/runtime/blob/06a300336080793370c7075ccf658abb70869126/src/coreclr/tools/aot/ILCompiler/README.md describes how .mibc file can be generated for a HelloiOS application.
This is very much a WIP but can be used as an input to continue working on other tasks/requirements.

I will create a PR once I have something more concrete.

PS Thanks @MichalStrehovsky for the initial work.

[kotlarmilos]

Good, it can be used as an input for testing PGO in Mono AOT.

I will create a PR once I have something more concrete.

Can it work with both Mono and Native AOT as a single tool?

[ivanpovazan]

Can it work with both Mono and Native AOT as a single tool?

Not at this point. Since our main objective is improving codegen for iOS, we will have to enable building ILCompiler for this platform or extract the dependency scanning feature into a separate tool which is platform agnostic. This is yet to be defined.

[MichalStrehovsky]

extract the dependency scanning feature into a separate tool which is platform agnostic

I consider this outcome more likely. The general shape of the dependency analysis is similar, but the dependency graph that NativeAOT is building is geared towards generating an object file for the NativeAOT runtime with any/all optimizations applicable to that model and I think there's going to be enough of small differences that we'll not want to use the exact same classes.

The more likely outcome is that we would be sharing some source files and potentially ILCompiler.DependencyAnalysis/ILCompiler.TypeSystem assemblies, but it would be a different tool at this point.

[ivanpovazan]

Status update

This is a status update on the progress regarding optimizing generics by using NativeAOT dependency analysis guide optimization (DAGO) with Mono in full AOT compilation. The following has been done:

• ILCompiler changes:

  • ILCompiler's dependency analysis performed by ILScanner has been adapted to work with Mono system module - System.Private.CoreLib
  • ILCompiler has been adapted to output the results of ILScanner (the list of methods required for the application to run) in a .mibc format (further referred as DAGO profile) - new compiler switches have been introduced to accomplish this
  • Additional control over which method types should be included in the DAGO profile is introduced - for example: we are interested mainly which specific instances of generic methods are needed during runtime, this can be accomplished by passing: --scanmibcforallgenerics

• Build system changes:

  • Minor adaptations of MonoAOTCompiler build task to prevent forcing profile-only switch passed to MonoAOT compiler - this means we are using the same approach as with PGO profiles
  • Improving build experience and testing for mono samples (HelloWorld - osx, HelloiOS - ios), that utilize the set up of running ILCompiler first to produce DAGO profile, which is then passed to MonoAOT compiler for compilation

• Mono changes:

  • Introduced Array<T> in Mono to enable ILScanner to produce a more accurate output
  • Disabled generation of GSHAREDVTs by default

Observations

The current setup works for HelloiOS sample and shows the following results:

Mode	Size (bytes)	Size (MB)	Saving
GSHAREDVT	27980822	27,98	N/A
noGSHAREDVT	26412238	26,41	-5,61%
DAGO+noGSHAREDVT	26734318	26,73	-4,45%

However, further inspection of how impactful would it be to apply the same principle on MAUI iOS applications showed some obstacles:

• With MAUI iOS applications turning off GSHAREDVTs with Mono actually increases the application size due to the following:

  runtime/src/mono/mono/mini/aot-compiler.c

  Line 5647 in 1cd8c2e

  if ((acfg->jit_opts & MONO_OPT_GSHAREDVT) && m_class_get_image (klass) == mono_defaults.corlib && mono_class_is_ginst (klass) && mono_class_get_generic_class (klass)->context.class_inst && is_vt_inst (mono_class_get_generic_class (klass)->context.class_inst) &&

  
  and similar paths in the compiler, where compiler generates extra specific instances instead.
• In general it seems that even though GSHAREDVTs are large and slow functions, they actually bring benefits to having a smaller application
• This is observable by compiling MAUI iOS template application with NativeAOT which shows a huge number of specific instances when Linq.Expressions assembly is compiled (for example: instances of System.Func`3 delegates that take up 25% of accountable size of the generated object file). This happens most probably due to incorrect compilation/trimming of Linq.Expressions assembly noted by @MichalStrehovsky comment.

Follow-up work

• Investigate whether this approach would change how the AOT compilers (both MonoAOT and NativeAOT) compile Linq.Expressions
• Come up with a heuristic when to use specific instances from DAGO profile over GSHAREDVTs during code generation, for example:
  • Special case Linq.Expressions (and similar nonAOT friendly patterns - eg: reflection) for which we always generate GSHAREDVTs, while for the rest we use what DAGO profile instructs
  • Use a intersection of PGO and DAGO profiles for generics, to improve startup performance but also to reduce size by generating specific instances from the intersection, and additionally GSHAREDVTs for all other cases (safe approach)
• Measure potential improvements on a MAUI iOS template application and decide whether it is feasible to pursue this optimization further

Final notes

• There are a few workarounds made for the scanning phase to work which need to be cleaned-up if we decide to invest into this feature. That work should not be too difficult.
• Note for moving forward by @MichalStrehovsky is that the proper way of pursuing this would be to pull ILScanner out of ILCompiler which would reuses the basic dependency engine, the type system, etc. Basically, using ILImporter.cs instead of ILImporter.Scanner.cs (which is more NativeAOT/RyuJIT specific)
• The state of this work is still experimental and more info can be found in: https://github.com/ivanpovazan/runtime/blob/nativeaot-mono/src/mono/sample/iOS/README.md (and in general through commits pushed to that branch).