optimizer: supports callsite annotations of inlining, fixes #18773

[aviatesk]

Enable @inline/@noinline annotations on function callsites.
From #40754.

Now @inline and @noinline can be applied to a code block and then
the compiler will try to (or try not to) inline calls within the block:

@inline f(...) # The compiler will try to inline `f`

@inline f(...) + g(...) # The compiler will try to inline `f`, `g` and `+`

@inline f(args...) = ... # Of course annotations on a definition is still allowed

Here are couple of notes on how those callsite annotations will work:

• callsite annotation always has the precedence over the annotation
  applied to the definition of the called method, whichever we use
  @inline/@noinline:

  @inline function explicit_inline(args...)
      # body
  end
  
  let
      @noinline explicit_inline(args...) # this call will not be inlined
  end

• when callsite annotations are nested, the innermost annotations has
  the precedence

  @noinline let a0, b0 = ...
      a = @inline f(a0)  # the compiler will try to inline this call
      b = notinlined(b0) # the compiler will NOT try to inline this call
      return a, b
  end

They're both tested and included in documentations.

Co-authored-by: Joseph Tan [email protected]

---

• closes callsite inlining #18773
• closes optimizer: support callsite annotations of @inline and @noinline #40754

[aviatesk]

I tried the approach to use uncached source when the callsite is forced to be inlined (a1b1435), but it turns out that then inlining will depend o local state of inference and so the first entry and the succeeding ones may generate different code, e.g. from a test case I added:

function multilimited(a)
    if @noinline(isType(a))
        return @inline(multilimited(a.parameters[1]))
    else
        return rand(Bool) ? rand(a) : @inline(multilimited(a))
    end
end

# output might be different if we depend on inference local state
code_typed(m.multilimited, (Any,))
code_typed(m.multilimited, (Any,))

We might be able to keep source so that it doesn't depend on the local state, but it could be tricky.
Rather I'd like to always just re-infer when a source is unavailable, at least in this first iteration.

/cc @vtjnash

[vtjnash]

I think this needs to happen during inference. It seems quite wrong to make a new NativeInterpreter here, and can probably have many other problems. We used to do some amount of recursion here, and we have gotten rid of it.

[aviatesk]

What do you think about fea41f2 ? We still need fresh optimization w/o inlining hack because those uncached sources are not optimized.

Well, now I'm wondering if it's really useful. I benchmarked some functions that hit this pass, and I couldn't find any cases where this "single-level-inlining" can help the performance.. Honestly I don't think it's worth the complexity that comes with this support.

[aviatesk]

#41328 (comment)

[tisztamo]

Does callsite @inline force inlining completely, ignoring the cost model? (Also close #40292?) (re-commenting after accidental delete)

[aviatesk]

Does callsite @inline force inlining completely, ignoring the cost model?

Callsite inlining will happen in regardless of the cost model, but not "force inlining completely", since there are some cases when a method body itself is unavailable (think when there are recursive calls but annotated as @inline).

I would appreciate if you could test this PR for your target, though.

[tisztamo]

Just tested and it works, so this also closes #40292. Thank you very much!

[aviatesk]

#42082 should fix the "serious" flaw with this PR, and now callsite inlining should work more reliably.

[chriselrod]

@oscardssmith If you haven't tried this out yet:

julia> function vexp!(y,x)
           @inbounds for i ∈ eachindex(y,x)
               y[i] = exp(x[i])
           end
       end
vexp! (generic function with 1 method)

julia> function vexp_inline!(y,x)
           @inbounds for i ∈ eachindex(y,x)
               y[i] = @inline exp(x[i])
           end
       end
vexp_inline! (generic function with 1 method)

julia> x = randn(256); y = similar(x);

julia> @benchmark vexp!($y, $x)
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
 Range (min … max):  1.234 μs …  2.336 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.329 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.334 μs ± 49.344 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

                       █▄                                    ▁
  ▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅██▁▇▃▁▆▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▃▁▁▁▁▁▆▃▄▄▇▆▅▆ █
  1.23 μs      Histogram: log(frequency) by time     1.49 μs <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark vexp_inline!($y, $x)
BenchmarkTools.Trial: 10000 samples with 540 evaluations.
 Range (min … max):  212.893 ns … 316.361 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     213.315 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   214.200 ns ±   6.480 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

   ▂▇█▇▃                   ▁▂▃▂▂▂▂                              ▂
  ▅█████▃▁▃▃▁▁▁▁▁▁▄▁▁▁▁▁▃▄▆████████▇██▇▇▇▆▅▄▃▅▄▅▃▁▄▄▃▄▄▄▄▄▅▃▄▄▅ █
  213 ns        Histogram: log(frequency) by time        220 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.