JIT: De-abstraction in .NET 10 #108913
Comments
AndyAyersMS
added
area-CodeGen-coreclr
dotnet-policy-service
bot
added
the
untriaged
|
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
dotnet-policy-service
bot
removed
the
untriaged
|
I worked up a suite of related benchmarks: dotnet/performance#4522. Ideally all these (save for the With recent main we see:
Note there is a penalty for enumerating an array inside another loop. That's something else we should try and fix.
|
Add some benchmarks for array deabstraction. See dotnet/runtime#108913
|
Seems like #102965 could be useful her (enhances local morph's ability to propagate addresses in loops). However, I will also need local morph to be more aggressive with handlers. Looks like in addition to per-loop summaries I'd need per-try summaries (at least for try/finally/fault), and if a local is not defined within a try, we can propagate any assertion about that local true at try entry to the try's handler. The computation of the summaries is simple enough, the question is mainly how to do this efficiently and whether it is generally useful or special purpose. It's possible that in |
|
@EgorBo points out that array interface methods (like int Test(ICollection<string> col) => col.Count;G_M15003_IG01: ;; offset=0x0000
sub rsp, 40
;; size=4 bbWeight=1 PerfScore 0.25
G_M15003_IG02: ;; offset=0x0004
mov rcx, rdx
mov r11, 0x7FFCCA100210 ; code for System.Collections.Generic.ICollection`1[System.__Canon]:get_Count():int:this
call [r11]System.Collections.Generic.ICollection`1[System.__Canon]:get_Count():int:this
nop
;; size=17 bbWeight=1 PerfScore 3.75
G_M15003_IG03: ;; offset=0x0015
add rsp, 40
ret The JIT is able to devirtualize (under GDV) but the resulting method can't be inlined, so we drop the GDV: (Note the runtime maps all ref types to If the JIT can learn that the collection type is In thes cases it seems like the JIT has all the information it needs to call the underlying method properly (and for |
The JIT recently enabled devirtualization of `GetEnumerator`, but other methods were inhibited from devirtualization because the runtime was returning an instantiating stub instead of the actual method. This blocked inlining and the JIT currently will not GDV unless it can also inline. So for instance `ICollection<T>.Count` would not devirtualize. We think we know enough to pass the right inst parameter (the exact method desc) so enable this for the array case, at least for normal jitting. For NAOT array devirtualization happens via normal paths thanks to `Array<T>` so should already fpr these cases. For R2R we don't do array interface devirt (yet). There was an existing field on `CORINFO_DEVIRTUALIZATION_INFO` to record the need for an inst parameter, but it was unused and so I renamed it and use it for this case. Contributes to dotnet#108913.
For empty arrays, `GetEnumerator` returns a static instance rather than a new instance. This hinders the JIT's ability to optimize when it is able to stack allocate the new instance. Detect when a stack allocation comes from an array `GetEnumerator` and fold the branch in the inlined enumerator method to always take the new instance path (since it is now cheap, allocation free, and is functionally equivalent). Contributes to dotnet#108913.
The actual distinction is whether the array type is shared or not... (eg Even if we enable GDV by finding inline info, we end up blocking inlining, because the |
For empty arrays, `GetEnumerator` returns a static instance rather than a new instance. This hinders the JIT's ability to optimize when it is able to stack allocate the new instance. Detect when a stack allocation comes from an array `GetEnumerator` and fold the branch in the inlined enumerator method to always take the new instance path (since it is now cheap, allocation free, and is functionally equivalent). Contributes to #108913.
#109209) The JIT recently enabled devirtualization of `GetEnumerator`, but for some cases were inhibited from devirtualization because the runtime was returning an instantiating stub instead of the actual method. This blocked inlining and the JIT currently will not GDV unless it can also inline. So for instance `ICollection<T>.Count` would not devirtualize. We think we know enough to pass the right inst parameter (the exact method desc) so enable this for the array case, at least for normal jitting. For NAOT array devirtualization happens via normal paths thanks to `Array<T>` so should already fpr these cases. For R2R we don't do array interface devirt (yet). There was an existing field on `CORINFO_DEVIRTUALIZATION_INFO` to record the need for an inst parameter, but it was unused and so I renamed it and use it for this case. Contributes to #108913.
|
With the recently merged PRs we're able to handle a broad range of cases where the JIT learns the collection is an array via its own data flow. For cases where the collection is likely an array there is still work to do... |
|
Spent some time looking at what it might take to get normal loop opts to kick in. The graph below shows the situation during loop inversion. We focus just on the loop.
The block colors:
Upstream from BB08 V17 is set to -1 and V18 to the array length. As is, the loop is not an obvious candidate for inversion; the exit test in BB13 is in a different block then the entry BB07. These can be merged but it takes a bit of work. First we need to tail duplicate BB13, then optimize the branches; that leaves something like
and now BB07 is both loop entry and exit, so can be duplicated:
We still have the "wraparound IV" case to sort out (V14,V17) but perhaps we can handle that nowSo the key seems to be to at least do the early flow opt where we tail duplicate the V13 test block (this is the vestige of the |
|
I generalize loop inversion in #109346, and some form of loop inversion does now kick in for the simple case, but it still doesn't seem to be enough for the remaining loop opts to kick in... in fact it seems we now can't eliminate the bounds check either. I haven't dug in deeper at what goes wrong, hopefully it's something simple. |
For empty arrays, `GetEnumerator` returns a static instance rather than a new instance. This hinders the JIT's ability to optimize when it is able to stack allocate the new instance. Detect when a stack allocation comes from an array `GetEnumerator` and fold the branch in the inlined enumerator method to always take the new instance path (since it is now cheap, allocation free, and is functionally equivalent). Contributes to dotnet#108913.
dotnet#109209) The JIT recently enabled devirtualization of `GetEnumerator`, but for some cases were inhibited from devirtualization because the runtime was returning an instantiating stub instead of the actual method. This blocked inlining and the JIT currently will not GDV unless it can also inline. So for instance `ICollection<T>.Count` would not devirtualize. We think we know enough to pass the right inst parameter (the exact method desc) so enable this for the array case, at least for normal jitting. For NAOT array devirtualization happens via normal paths thanks to `Array<T>` so should already fpr these cases. For R2R we don't do array interface devirt (yet). There was an existing field on `CORINFO_DEVIRTUALIZATION_INFO` to record the need for an inst parameter, but it was unused and so I renamed it and use it for this case. Contributes to dotnet#108913.
De-Abstraction
In .NET 10 we hope to further enhance the JIT's ability to remove abstraction overhead from code.
Stack Allocation Improvements
See #104936
During .NET 10 we would like to implement 2-3 enhancements to stack allocation of ref class instances. Priority may be given to issues that enable array de-abstraction (see below).
Delegate GDV Improvements
We currently only GDV instance delegates. We'd like to extend support to static delegates.
PGO Improvements
We currently lose a bit of performance in R2R compiled methods with Tiered PGO, because the instrumented version of the method doesn't collect profile data for inlinees.
#44372
Inlining Improvements
We'd like to enable inlining of methods with EH.
#108900
Array Enumeration De-Abstraction
Completed Work:
Todo:
Background
The goal of this work is to (to the best of our ability) eliminate the abstraction penalty for cases where an
IEnumerable<T>is iterated viaforeach, and the underlying collection is an array (or is very likely to be an array).In previous releases we've built a number of optimizations that can reduce abstraction overhead. But there is still a lot of room for improvement, especially in cases like the above, where the abstraction pattern involves several abstract objects acting in concert.
What is the Abstraction Penalty?
Consider the following pair of benchmark methods that both sum up an integer array:
These two methods do the exact same computation, yet benchmarking shows the second method takes 4.5x as long as the first (with 512 element arrays, using very early .NET 10 bits incorporating #108604 and #108153):
This sort of overhead from an abstract presentation of computation is commonly known as the abstraction penalty.
Note things used to be far worse; .NET 6's ratio here is 12.6.
Why is there an abstraction penalty?
The IL generated for the
foreach_static_readonly_array_via_interfaceis expressed in the shape of the abstract enumeration pattern: firste.GetEnumerator()is called on the abstract collection to produce an abstract enumerator, and then loop iterates viaMoveNext()andget_Current()interface calls on this enumerator, all wrapped in a try finally to properly dispose the enumerator should an exception arise.Seeing through all this to the actual simple computation going on in the loop requires a surprising amount of optimization machinery. In past releases we've built many of the necessary pieces, and now it's time to get them all working together to remove the remaining overhead.
In particular we need to leverage:
More generally the JIT will need to rely on PGO to determine the (likely) underlying type for the collection.
Why focus on Arrays?
Arrays are the most common and also the simplest collection type. Assuming all goes well we may try and stretch the optimization to cover Lists.
What needs to be done?
When the collection is an array, the enumerator is an instance of the ref class
SZGenericArrayEnumerator<T>. Thanks to #108153 we can devirtualize (under guard) and inline the enumerator constructor, and devirtualize and inline calls on the enumerator. And in some cases we can even stack allocate the enumerator (note in the table above, .NET 10 no longer has allocations for the benchmarks).Current inner loop codegen:
However, we cannot yet fully optimize the enumeration loop:
SZGenericArrayEnumeratorconstructor has an optimization for empty arrays, where instead of constructing a new enumerator instance, it returns a static instance. So at the enumerator use sites there is some ambiguity about which object is enumerating. For cases where the array length is known this ambiguity gets resolved, but too late in the phase order.SZGenericArrayEnumerator, so all three definitions can reach through the enumerator GDV tests (see JIT: Support for devirtualizing array interface methods #108153 (comment) for a picture and more notes). And we may get confused by the try/finally or try/fault which will also contain a reference to the enumerator (for GDV).While these may seem like small problems, the solutions are not obvious. Either we need to disentangle the code paths for each possibility early (basically do an early round of cloning, not just of the enumeration loop but of all the code from the enumerator creation sites to the last use of the enumerator and possibly the EH regions) or we need to think about making our escape and address propagation logic be flow-sensitive and contextual and introduce runtime disambiguation for the reaching values (that is, at each enumerator use site, we have to test if the enumerator is the just-allocated instance from the
SZGenericArrayEnumeratorthat we hope to stack allocate and promote).The cloning route seems more viable, but it is costly to duplicate all that code and we'll have to do it well in advance of knowing whether the rest of the optimizations pay off. So we might need to be able to undo it if there's no real benefit.
The text was updated successfully, but these errors were encountered: