Very slow compiler performance with long function and many list comprehensions

[GlenHertz]

Hi,

I'm tried to strip down an example to something small but I was left with a function that makes no sense but shows the performance issue. It takes about 4 seconds and half a GB of memory to compile this function. The testcase is available here:

https://gist.github.com/GlenHertz/11387643#file-slow_performance-jl

I tried to add type annotations to speed things up but it didn't help.

Thanks,

Glen

[JeffBezanson]

Are the types with lots of fields relevant to this effect, or is it just the comprehensions?

[GlenHertz]

I don't think the types are relevant. It seems to be a combination of a
long function body (if I split it into smaller functions it helps) and the
comprehensions.

On Tue, Apr 29, 2014 at 4:04 PM, Jeff Bezanson [email protected]:

Are the types with lots of fields relevant to this effect, or is it just
the comprehensions?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/6685#issuecomment-41725058
.

[vtjnash]

Here's a version that only has one field. It's a little longer and a little faster, but still shows super-linear (super-quadratic?) growth in type-inference time.

The parse time of this function is also very bad (12 seconds).

https://gist.github.com/vtjnash/27ff622b2cbed22b51dc

Perhaps the number of SSA variables (1898) is causing issues?

As a random side note, when using the type with with 80-fields, type inference takes about 10% longer (e.g. 4.5 instead of 4 seconds) if all of the comprehensions are on f80 instead of f1.

(ps. timings were done on my jn/inline_partialmatch branch)

[quinnj]

Has this been helped by the work in #7977?

[JeffBezanson]

Nope. It's just that the function has 3000 statements and over a thousand variables.

[KristofferC]

This takes 2 seconds to compile and 80 MB memory. Can this be closed?

[vtjnash]

Yes. Would be nice to use examples like this to start adding compile and parse time tests to Nanosoldier's BaseBenchmarks.jl.

[KristofferC]

@time println(t)  # about 4 sec, 550MB

is now

 1.365269 seconds (2.69 M allocations: 147.406 MiB, 5.61% gc time)

Not sure what the criteria is to close this so I will just do that. Feel free to reopen.