Backtraces: much better, but still want more

[timholy]

I'm testing this on branch yyc/codegen/deopt-tls from #17800, so that julia --inline=no doesn't segfault, but master gives the same bad version of the backtrace when inlining is on. Discovered this while working on #17816.

tim@cannon:~/src/julia-0.5$ ./julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-dev.5 (2016-08-04 06:09 UTC)
 _/ |\__'_|_|_|\__'_|  |  yyc/codegen/deopt-tls/f90eab3* (fork: 1 commits, 0 days)
|__/                   |  x86_64-linux-gnu

julia> using OffsetArrays

julia> v = OffsetArray(rand(3), -1:1)
OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}} with indices (-1:1,):
 0.593194
 0.510626
 0.633172

julia> v'
ERROR: size not supported for arrays with indices (-1:1,); see http://docs.julialang.org/en/latest/devdocs/offset-arrays/
 in errmsg(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at /home/tim/.julia/v0.6/OffsetArrays/src/OffsetArrays.jl:40
 in collect(::Base.Generator{Base.Prod2{UnitRange{Int64},OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}},Base.##207#208}) at ./array.jl:275
 in ctranspose(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at ./arraymath.jl:273

Now compare what happens when we turn inlining off:

tim@cannon:~/src/julia-0.5$ ./julia --inline=no
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-dev.5 (2016-08-04 06:09 UTC)
 _/ |\__'_|_|_|\__'_|  |  yyc/codegen/deopt-tls/f90eab3* (fork: 1 commits, 0 days)
|__/                   |  x86_64-linux-gnu

julia> using OffsetArrays

julia> v = OffsetArray(rand(3), -1:1)
OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}} with indices (-1:1,):
 0.204115 
 0.594476 
 0.0276101

julia> v'
ERROR: size not supported for arrays with indices (-1:1,); see http://docs.julialang.org/en/latest/devdocs/offset-arrays/
 in error(::String) at ./error.jl:21
 in errmsg(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at /home/tim/.julia/v0.6/OffsetArrays/src/OffsetArrays.jl:40
 in size(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at /home/tim/.julia/v0.6/OffsetArrays/src/OffsetArrays.jl:41
 in _prod_size(::UnitRange{Int64}, ::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}, ::Base.HasShape, ::Base.HasShape) at ./iterator.jl:322
 in size(::Base.Prod2{UnitRange{Int64},OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}}) at ./iterator.jl:311
 in _array_for(::Type{T}, ::Base.Prod2{UnitRange{Int64},OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}}, ::Base.HasShape) at ./array.jl:262
 in collect(::Base.Generator{Base.Prod2{UnitRange{Int64},OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}},Base.##207#208}) at ./array.jl:275
 in transpose(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at ./arraymath.jl:275
 in ctranspose(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at ./arraymath.jl:273
 in eval(::Module, ::Any) at ./boot.jl:234

which is vastly more useful.

I have the impression that we were in better shape a few weeks ago, but I haven't tested.

[JeffBezanson]

What happens with inlining and -O0?

[timholy]

No better.

[vtjnash]

Inlining can't preserve line numbers on the return value without a more linear IR

[JeffBezanson]

As long as this isn't a regression w.r.t. 0.4 or 0.5 1-2 months ago then this is acceptable for the time being.

[timholy]

I tried to build a version from a month or so ago, but ran into very persistent libssh2 problems, even with a make distclean.

[tkelman]

make -C deps distclean-libgit2 distclean-libssh2 distclean-mbedtls since there were some commit ranges where that stack was not robust to rebuilds

[tkelman]

the top level make distclean also doesn't do that much. make distcleanall is the one that will distclean all deps too.

[timholy]

Thanks, @tkelman. I randomly picked dad6836 from about a month ago, and it's no better:

julia> using OffsetArrays

julia> v = OffsetArray(rand(3), -1:1);

julia> v'
ERROR: size not supported for arrays with indices (-1:1,); see http://docs.julialang.org/en/latest/devdocs/offset-arrays/
 in errmsg(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at /home/tim/.julia/v0.5/OffsetArrays/src/OffsetArrays.jl:40
 in transpose at ./arraymath.jl:0 [inlined]
 in ctranspose(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at ./arraymath.jl:323
 in eval(::Module, ::Any) at ./boot.jl:234
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

It seems that comprehensions pose some problems for backtraces:

julia> [x for x in v]
ERROR: size not supported for arrays with indices (-1:1,); see http://docs.julialang.org/en/latest/devdocs/offset-arrays/
 in errmsg(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at /home/tim/.julia/v0.5/OffsetArrays/src/OffsetArrays.jl:40
 in length(::OffsetArrays.OffsetArray{Float64,1,Array{Float64,1}}) at ./abstractarray.jl:71
 in anonymous at ./<missing>:?
 in eval(::Module, ::Any) at ./boot.jl:234
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

For the future this might be worth looking into.

(All of these errors are fixed on #17816, so when that merges it will be harder to debug this.)

[KristofferC]

I think backtraces have been improved and rewritten enough for the current issue to be stale (the examples doesn't repro either). Can always open a new issue with more backtrace gripes.