some updates and fixes to inlining cost

[JeffBezanson]

• Instead of always inlining functions marked @inline, increase the cost threshold 10x 20x
• Don't inline functions inferred not to return
• statement_cost no longer needs to look at nested Exprs in general
• Fix cost of :copyast

This also fixes the regression in mapslices (#27417). It was due to inlining a function marked @inline, even though the argument types were abstract and the body contained many dynamic calls. I believe most compilers just increase the threshold (by a lot) for functions marked inline, and have a separate always_inline to truly force inlining. We should do the same thing; otherwise we end up inlining silly things the user is unlikely to have intended.

[JeffBezanson]

@nanosoldier runbenchmarks(ALL, vs = ":master")

[KristofferC]

I believe most compilers just increase the threshold (by a lot) for functions marked inline, and have a separate always_inline to truly force inlining.

Can we get that always_inline then? Because when the inliner gets it "wrong", the effect can be dramatic and not having an escape hatch seems like a step backwards.

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[JeffBezanson]

Yes, you can have always_inline, but beware: if we can't infer the types of arguments (f(x::Any)), we will inline the function anyway, with everything in it dynamically dispatched, instead of dispatching to an optimized version. That is a dramatic effect, and probably not what you want.

[timholy]

I suppose another alternative is to aggressively eliminate many current @inline annotations, and save the current @inline to mean always_inline. The data in #22210 (comment) seem to suggest that might be feasible.

But there's a lot of sense in this proposal.

[timholy]

What's the motivation for making this non-recursive?

[JeffBezanson]

The structure of the IR has changed such that calls no longer contain other calls, so it's just unnecessary.

[JeffBezanson]

Performance summary: some nice speedups from avoiding inlining things with dynamic dispatches, but some big slowdowns in find functions. The slowdowns were mostly related to the use of invoke in deprecations, so I just changed that.

[JeffBezanson]

@nanosoldier runbenchmarks(ALL, vs = ":master")

[nanosoldier]

Something went wrong when running your job:

NanosoldierError: failed to run benchmarks against primary commit: failed process: Process(`sudo cset shield -e su nanosoldier -- -c ./benchscript.sh`, ProcessExited(1)) [1]

Logs and partial data can be found here
cc @ararslan

[JeffBezanson]

@nanosoldier runbenchmarks(ALL, vs = ":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[JeffBezanson]

Fixes #26446

[JeffBezanson]

@nanosoldier runbenchmarks("collection" && "queries & updates", vs = ":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[JeffBezanson]

@nanosoldier runbenchmarks("collection" && "queries & updates", vs = ":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[JeffBezanson]

@nanosoldier runbenchmarks("string" && "readuntil", vs = ":master")

[nanosoldier]

Your benchmark job has completed - no performance regressions were detected. A full report can be found here. cc @ararslan

[vtjnash]

This branch should be dead-code

(also, it's available as the singleton_type function)

[JeffBezanson]

This might indicate a problem elsewhere, but it's not dead code --- in fact I needed this to fix a regression in an earlier version of the PR. A call to Core.sizeof had typeof(Core.sizeof) as the type of the function.

[vtjnash]

cost = plus_saturate(cost, ...)

[KristofferC]

Bump for @forceinline...

Issues like JuliaArrays/StaticArrays.jl#494 are quite frustrating when you always have to worry that the threshold might have gotten reached.

[JeffBezanson]

There might be something wrong with our cost model in this case --- a few dozen arithmetic ops should not be a problem. I'm worried about dynamic dispatches that take thousands of times longer than that. We can add forceinline as well though.

[KristofferC]

The example there has a SIMD loop though so might be why it doesn't inline. Anyway, having a way to force would be good in the inevitable cases where the cost model gets it wrong.

[vtjnash]

If the function type is maximally typed (isdispatchtuple(specTypes)), we could simply always obey the user annotation. In that case, you're already then at least guaranteed that keeping the dispatch barrier won't improve type information.