Failure to compile (hang in effect_free)

[timholy]

For me, the following fails to compile:

using DualNumbers

function myfunction{T}(ucomp::Array{T}, isimplex::AbstractVector{Int}, coord::Array{T})
    i1 = isimplex[1]
    i2 = isimplex[2]
    i3 = isimplex[3]
    i4 = isimplex[4]
    f1 = ucomp[1, i1] + coord[1, 1]
    g1 = ucomp[2, i1] + coord[2, 1]
    h1 = ucomp[3, i1] + coord[3, 1]
    f2 = ucomp[1, i2] + coord[1, 2]
    g2 = ucomp[2, i2] + coord[2, 2]
    h2 = ucomp[3, i2] + coord[3, 2]
    f3 = ucomp[1, i3] + coord[1, 3]
    g3 = ucomp[2, i3] + coord[2, 3]
    h3 = ucomp[3, i3] + coord[3, 3]
    f4 = ucomp[1, i4] + coord[1, 4]
    g4 = ucomp[2, i4] + coord[2, 4]
    h4 = ucomp[3, i4] + coord[3, 4]
    V = f1*g3*h2 - f1*g2*h3 + f2*g1*h3 - f2*g3*h1 - f3*g1*h2 + f3*g2*h1 + f1*g2*h4 - f1*g4*h2 - f2*g1*h4 + f2*g4*h1 + f4*g1*h2 - f4*g2*h1 - f1*g3*h4 + f1*g4*h3 + f3*g1*h4 - f3*g4*h1 - f4*g1*h3 + f4*g3*h1 + f2*g3*h4 - f2*g4*h3 - f3*g2*h4 + f3*g4*h2 + f4*g2*h3 - f4*g3*h2
end

unew = 2rand(2, 3, 3) .- 1
unewd = convert(Array{Dual{Float64}}, unew)
# unewd = convert(Array{Float64}, unew)
println("have unewd")

thiscoord = Array(Dual{Float64}, 3, 4)
# thiscoord = Array(Float64, 3, 4)
isimplex = [1:4]
myfunction(unewd, isimplex, thiscoord)

println("call to myfunction succeeded")

If you uncomment the two lines that declare the arrays as Float64 rather than Dual{Float64}, then it compiles fine. (It gives a BoundsError, but I'm not worried about that---this is a massively stripped-down example from the real code that triggered this.)

When I include this file, it just hangs. If I then hit Ctrl-C, the error I get is

^CERROR: interrupt
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1895
 in effect_free at inference.jl:1886
 in effect_free at inference.jl:1879
 in effect_free at inference.jl:1886
 in inlineable at inference.jl:2091
 in inlining_pass at inference.jl:2382
 in inlining_pass at inference.jl:2422 (repeats 6 times)
 in inlining_pass at inference.jl:2298
 in typeinf at inference.jl:1358
 in typeinf at inference.jl:1497
 in typeinf_ext at inference.jl:1186

One thing I should add is that with earlier versions, as I was progressively removing code not needed to demonstrate this problem, I sometimes found that it would suddenly work if I re-included the file enough times. I haven't seen that with this version, but just FYI.

julia> versioninfo()
Julia Version 0.3.0-prerelease+2658
Commit 752dc20* (2014-04-18 11:35 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7 CPU       L 640  @ 2.13GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY)
  LAPACK: libopenblas
  LIBM: libopenlibm

[timholy]

I started doing a git bisect, and the problem went away and I didn't find a way to make it come back. Sigh.

[timholy]

Oh wait, I was too hasty to close: the reason it went away was that I had left the file in a state where I uncommented the Float64 lines (duh).

git bisect identifies 377904b as the first bad commit.

[vtjnash]

shorting the following line helps to isolate the issue:

    V = f1*g3*h2 - f1*g2*h3 + f2*g1*h3 - f2*g3*h1 - f3*g1*h2 + f3*g2*h1 + f1*g2*h4 - f1*g4*h2 - f2*g1*h4 + f2*g4*h1 + f4*g1*h2 - f4*g2*h1 - f1*g3*h4 + f1*g4*h3 

we are inlining extremely far beyond the realm of reasonable amounts of inlining. the inliner is going to extremely great lengths (copying Exprs) to keep everything in a single line

[timholy]

Thanks for the fix, Jameson. Are you saying that entire function was being inlined? In contrast to my well-known views about the importance of inlining next 😄, here that does indeed seem to be quite unnecessary. I confess I don't understand nearly enough about the internals to understand why that would have caused compilation to hang. Are you sure this isn't yet another band-aid (different from #6569) over a deeper problem?

[vtjnash]

it didn't hang, but would have taken several hours to finish. the compiler had created a one-liner with many tens (or hundreds) of thousands of expressions

i altered the inline_worthy function to care more about the actual content (expected number of emitted instructions), rather than the number of lines (which was a measure of the complexity of the inliner required to inline it)

this is both a band-aid, and a correct solution. the inline_worthy test is more correct now. however, the inliner should probably be more willing to create local variables, over making a large number of repeated copies of an (effect-free) expression, if the expression is large.

[JeffBezanson]

Maybe we should have a size threshold for introducing temp vars for an expression.