inference: IPO constant propagation

[vtjnash]

Implements inter-procedural constant propagation.

fixes #5560
fixes #24011
fixes #14324
(based on some commits in #24337)

[StefanKarpinski]

Well, that's a terrifyingly large diff to inference, but it's a very nice optimization to have!

[oscardssmith]

It would be great to have nandolier on this type of thing

[yuyichao]

It would be great to have nandolier on this type of thing

It's it more like "It would be great to have nandolier"

[JeffBezanson]

Looks pretty good 👍

How long do those cache arrays get?

[vtjnash]

How long do those cache arrays get?

During sysimg build, the largest was 4728 items. The average size during a lookup was 470 items. I didn't collect any other stats (like the number of times we entered inference) for comparison though.

[vtjnash]

@nanosoldier runbenchmarks(ALL, vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[JeffBezanson]

It'd be good to have the runtime intrinsics fixes in a separate PR and merge that first.

[vtjnash]

Sure. Other than that, this is all ready to go. (passing all tests and nanosoldier is happy – the noise in the sparse test appears to also happens on master locally)

[vtjnash]

I guess I'll merge this tomorrow, unless there's any requests to defer it.

[vtjnash]

Would have been really nice if nanosoldier nightlies were working first though.

[JeffBezanson]

The compiler performance impact of this is surprisingly small, but there's still about a 30 second regression in sysimg build time for me (5'16", while master is about 4'45" and 0.6 is < 3'). We should find some way to moderate this.

[vtjnash]

Yeah, I was mildly surprised too, since I didn't add any particular limits on it (but I had some expectations from past analysis). How much do you want it to back off on attempting to do better optimization?

[vtjnash]

Since I find that timing the sysimg build is often not a representative workload, I've also timed running all of the tests in sequence. Looks like the net impact (on this inference heavy workload) is about 18%, so I think this is good to merge now.

master:
7808.731699 seconds (2.61 M allocations: 131.345 MiB, 0.00% gc time)
PR:
9249.809467 seconds (4.68 M allocations: 230.148 MiB, 0.00% gc time)

[JeffBezanson]

The sysimg build time regression described above is about 11%, so that's worse. I'm not convinced the trade-off is worthwhile as-is. There is definitely some code out there that benefits from this, but most of the code we're testing here probably doesn't, yet pays a hefty 18% cost.

[vtjnash]

Alright, I'll stop working on improving inference's capabilities here. Should we close the existing bug reports (#5560, #24011, and #14324) as wont-fix?

[oscardssmith]

How difficult would it be to place a limit where we only propagate "easy" cases. My guess is that this would yield much of the benefit, but might let us bail out earlier in most cases. Alternatively, could IPO be opt in?

[vchuravy]

Is this good to be merged?

[ararslan]

@nanosoldier runbenchmarks(ALL, vs=":master")

[vtjnash]

We can’t merge any features while CI is broken, but yes this should be ready to go. Hopefullly folks deal with the CI issues soon and quickly, as that’s likely to delay our desired freeze date.

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[JeffBezanson]

CI is now relatively healthy. Rebase, test, and merge?

[vtjnash]

rebased and CI looks happy with this

[vchuravy]

Let's ask nanosoldier one last time?
@nanosoldier runbenchmarks(ALL, vs=":master")

[pabloferz]

It seems that nanosoldier didn't start

@nanosoldier runbenchmarks(ALL, vs=":master")

[ararslan]

The Nanosoldier webhook log on this repo shows a 400 from both of your requests. I have no idea why but maybe third time's a charm?

@nanosoldier runbenchmarks(ALL, vs=":master")

[ararslan]

Nanosoldier gave me a 400 as well so I put him in a time out (AKA I restarted the server).

@nanosoldier runbenchmarks(ALL, vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[quinnj]

woohoo!