Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unnecessary allocations and illegal instruction crash with broadcasting #482

Closed
schmrlng opened this issue Aug 31, 2018 · 4 comments
Closed

Comments

@schmrlng
Copy link
Contributor

This is probably a base julia bug, but maybe we're doing something improper with broadcasting? The crash below happens consistently for me on the second @benchmark run.

  | | |_| | | | (_| |  |  Version 1.0.0 (2018-08-08)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |

julia> versioninfo()
Julia Version 1.0.0
Commit 5d4eaca0c9 (2018-08-08 20:58 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: AMD Ryzen 7 1800X Eight-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, znver1)

(v1.0) pkg> st
    Status `~/.julia/environments/v1.0/Project.toml`
  [6e4b80f9] BenchmarkTools v0.4.0
  [90137ffa] StaticArrays v0.8.3

julia> using LinearAlgebra

julia> using StaticArrays

julia> using BenchmarkTools

julia> perp(v) = SVector(v[2], -v[1])
perp (generic function with 1 method)

julia> X = rand(SVector{10,SVector{2,Float64}})
10-element SArray{Tuple{10},SArray{Tuple{2},Float64,1,2},1,10}:
 [0.950379, 0.824148]
 [0.555687, 0.25037] 
 [0.767836, 0.180282]
 [0.221228, 0.511591]
 [0.603886, 0.886817]
 [0.465518, 0.755331]
 [0.683507, 0.488349]
 [0.395891, 0.613276]
 [0.0759758, 0.77378]
 [0.530958, 0.711048]

julia> f(X) = perp.(X) ./ norm.(X)
f (generic function with 1 method)

julia> @benchmark f($X)
BenchmarkTools.Trial: 
  memory estimate:  1.17 KiB
  allocs estimate:  4
  --------------
  minimum time:     108.222 ns (0.00% GC)
  median time:      137.398 ns (0.00% GC)
  mean time:        159.887 ns (12.57% GC)
  maximum time:     31.702 μs (99.42% GC)
  --------------
  samples:          10000
  evals/sample:     949

julia> @benchmark f($X)
Unreachable reached at 0x7f801417941a

signal (4): Illegal instruction
in expression starting at no file:0
copy at /home/schmrlng/.julia/packages/StaticArrays/Ze5H3/src/broadcast.jl:24 [inlined]
materialize at ./broadcast.jl:724 [inlined]
f at ./REPL[7]:1 [inlined]
##core#366 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:293
##sample#367 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:299
#_run#4 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:327
unknown function (ip: 0x7f80141791db)
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-1.0/src/gf.c:1829
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
inner at ./none:0
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-1.0/src/gf.c:1829
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
jl_apply at /home/schmrlng/code/oss/julia-1.0/src/julia.h:1536 [inlined]
jl_f__apply at /home/schmrlng/code/oss/julia-1.0/src/builtins.c:556
jl_f__apply_latest at /home/schmrlng/code/oss/julia-1.0/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:690 [inlined]
#invokelatest at ./none:0 [inlined]
#run_result#16 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:32 [inlined]
#run_result at ./none:0 [inlined]
#run#18 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:46
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-1.0/src/gf.c:1829
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
warmup at ./none:0
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-1.0/src/gf.c:1829
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
do_call at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:324
eval_value at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:428
eval_stmt_value at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:363 [inlined]
eval_body at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:686
jl_interpret_toplevel_thunk_callback at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:799
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7f801ddf44bf)
unknown function (ip: 0x9)
jl_interpret_toplevel_thunk at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:808
jl_toplevel_eval_flex at /home/schmrlng/code/oss/julia-1.0/src/toplevel.c:787
jl_toplevel_eval_in at /home/schmrlng/code/oss/julia-1.0/src/builtins.c:622
eval at ./boot.jl:319
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
eval_user_input at /home/schmrlng/code/oss/julia-1.0/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:85
macro expansion at /home/schmrlng/code/oss/julia-1.0/usr/share/julia/stdlib/v1.0/REPL/src/REPL.jl:117 [inlined]
#28 at ./task.jl:259
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
jl_apply at /home/schmrlng/code/oss/julia-1.0/src/julia.h:1536 [inlined]
start_task at /home/schmrlng/code/oss/julia-1.0/src/task.c:268
unknown function (ip: 0xffffffffffffffff)
Allocations: 56019376 (Pool: 56015876; Big: 3500); GC: 562
[1]    7277 illegal hardware instruction (core dumped)  julia
@schmrlng
Copy link
Contributor Author

Same story on current master, for completeness:

  | | |_| | | | (_| |  |  Version 1.1.0-DEV.144 (2018-08-30)
 _/ |\__'_|_|_|\__'_|  |  Commit dabf70e6fe (0 days old master)
|__/                   |

julia> versioninfo()
Julia Version 1.1.0-DEV.144
Commit dabf70e6fe (2018-08-30 18:40 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: AMD Ryzen 7 1800X Eight-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, znver1)

(v1.1) pkg> st
    Status `~/.julia/environments/v1.1/Project.toml`
  [6e4b80f9] BenchmarkTools v0.4.0
  [90137ffa] StaticArrays v0.8.3

julia> using LinearAlgebra

julia> using StaticArrays

julia> using BenchmarkTools

julia> perp(v) = SVector(v[2], -v[1])
perp (generic function with 1 method)

julia> X = rand(SVector{10,SVector{2,Float64}})
10-element SArray{Tuple{10},SArray{Tuple{2},Float64,1,2},1,10}:
 [0.392512, 0.794166]
 [0.763505, 0.157252]
 [0.521086, 0.562327]
 [0.308199, 0.858053]
 [0.989704, 0.231244]
 [0.110478, 0.675843]
 [0.427602, 0.428346]
 [0.767848, 0.184462]
 [0.369643, 0.998114]
 [0.487112, 0.660135]

julia> f(X) = perp.(X) ./ norm.(X)
f (generic function with 1 method)

julia> @benchmark f($X)
BenchmarkTools.Trial: 
  memory estimate:  1.17 KiB
  allocs estimate:  4
  --------------
  minimum time:     109.667 ns (0.00% GC)
  median time:      134.751 ns (0.00% GC)
  mean time:        156.357 ns (12.65% GC)
  maximum time:     30.816 μs (99.26% GC)
  --------------
  samples:          10000
  evals/sample:     943

julia> @benchmark f($X)
Unreachable reached at 0x7f96cd03059a

signal (4): Illegal instruction
in expression starting at no file:0
copy at /home/schmrlng/.julia/packages/StaticArrays/Ze5H3/src/broadcast.jl:24 [inlined]
materialize at ./broadcast.jl:748 [inlined]
f at ./REPL[8]:1 [inlined]
##core#366 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:293
##sample#367 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:299
#_run#4 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:327
unknown function (ip: 0x7f96cd03035b)
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-master/src/gf.c:1841
jl_apply_generic at /home/schmrlng/code/oss/julia-master/src/gf.c:2196
inner at ./none:0
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-master/src/gf.c:1841
jl_apply_generic at /home/schmrlng/code/oss/julia-master/src/gf.c:2196
jl_apply at /home/schmrlng/code/oss/julia-master/src/julia.h:1558 [inlined]
jl_f__apply at /home/schmrlng/code/oss/julia-master/src/builtins.c:556
jl_f__apply_latest at /home/schmrlng/code/oss/julia-master/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:690 [inlined]
#invokelatest at ./none:0 [inlined]
#run_result#16 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:32 [inlined]
#run_result at ./none:0 [inlined]
#run#18 at /home/schmrlng/.julia/packages/BenchmarkTools/vesay/src/execution.jl:46
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-master/src/gf.c:1841
jl_apply_generic at /home/schmrlng/code/oss/julia-master/src/gf.c:2196
warmup at ./none:0
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-master/src/gf.c:1841
jl_apply_generic at /home/schmrlng/code/oss/julia-master/src/gf.c:2196
do_call at /home/schmrlng/code/oss/julia-master/src/interpreter.c:324
eval_value at /home/schmrlng/code/oss/julia-master/src/interpreter.c:430
eval_stmt_value at /home/schmrlng/code/oss/julia-master/src/interpreter.c:363 [inlined]
eval_body at /home/schmrlng/code/oss/julia-master/src/interpreter.c:688
jl_interpret_toplevel_thunk_callback at /home/schmrlng/code/oss/julia-master/src/interpreter.c:803
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7f96b8e342df)
unknown function (ip: 0x9)
jl_interpret_toplevel_thunk at /home/schmrlng/code/oss/julia-master/src/interpreter.c:812
jl_toplevel_eval_flex at /home/schmrlng/code/oss/julia-master/src/toplevel.c:792
jl_toplevel_eval_in at /home/schmrlng/code/oss/julia-master/src/builtins.c:622
eval at ./boot.jl:319
jl_apply_generic at /home/schmrlng/code/oss/julia-master/src/gf.c:2196
eval_user_input at /home/schmrlng/code/oss/julia-master/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:85
macro expansion at /home/schmrlng/code/oss/julia-master/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:117 [inlined]
#28 at ./task.jl:259
jl_apply_generic at /home/schmrlng/code/oss/julia-master/src/gf.c:2196
jl_apply at /home/schmrlng/code/oss/julia-master/src/julia.h:1558 [inlined]
start_task at /home/schmrlng/code/oss/julia-master/src/task.c:271
unknown function (ip: 0xffffffffffffffff)
Allocations: 55894048 (Pool: 55890429; Big: 3619); GC: 560
[1]    29696 illegal hardware instruction (core dumped)  ./julia

@schmrlng
Copy link
Contributor Author

Without needing BenchmarkTools to trigger the illegal instruction:

# test.jl
using LinearAlgebra
using StaticArrays
import StaticArrays: SUnitRange

perp(v) = SVector(v[2], -v[1])

struct A{N,T}
    v::SVector{N,SVector{2,T}}

    function A{N,T}(X) where {N,T}
        new(normalize.(perp.(X)))
    end
end
A(X) = A{length(X),eltype(eltype(X))}(X)

X = rand(SVector{3,SVector{2,Float64}})
@show typeof(X)
@show typeof(perp.(X))
@show typeof(normalize.(perp.(X)))
A(X)

resulting in

schmrlng@europa:/tmp$ julia test.jl
typeof(X) = SArray{Tuple{3},SArray{Tuple{2},Float64,1,2},1,3}
typeof(perp.(X)) = SArray{Tuple{3},SArray{Tuple{2},Float64,1,2},1,3}
typeof(normalize.(perp.(X))) = SArray{Tuple{3},SArray{Tuple{2},Float64,1,2},1,3}
Unreachable reached at 0x7fd47d2f778c

signal (4): Illegal instruction
in expression starting at /tmp/test.jl:20
copy at /home/schmrlng/.julia/packages/StaticArrays/Ze5H3/src/broadcast.jl:24 [inlined]
materialize at ./broadcast.jl:724 [inlined]
Type at /tmp/test.jl:11
Type at /tmp/test.jl:14
unknown function (ip: 0x7fd47d2f76c6)
jl_fptr_trampoline at /home/schmrlng/code/oss/julia-1.0/src/gf.c:1829
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
do_call at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:324
eval_value at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:428
eval_stmt_value at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:363 [inlined]
eval_body at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:686
jl_interpret_toplevel_thunk_callback at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:799
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7fd48f536bcf)
unknown function (ip: 0xffffffffffffffff)
jl_interpret_toplevel_thunk at /home/schmrlng/code/oss/julia-1.0/src/interpreter.c:808
jl_toplevel_eval_flex at /home/schmrlng/code/oss/julia-1.0/src/toplevel.c:787
jl_parse_eval_all at /home/schmrlng/code/oss/julia-1.0/src/ast.c:838
jl_load at /home/schmrlng/code/oss/julia-1.0/src/toplevel.c:821
include at ./boot.jl:317 [inlined]
include_relative at ./loading.jl:1038
include at ./sysimg.jl:29
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
exec_options at ./client.jl:229
_start at ./client.jl:421
jl_apply_generic at /home/schmrlng/code/oss/julia-1.0/src/gf.c:2182
unknown function (ip: 0x5626c6934b0b)
unknown function (ip: 0x5626c6934503)
__libc_start_main at /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
unknown function (ip: 0x5626c69345a9)
Allocations: 7277957 (Pool: 7276371; Big: 1586); GC: 15
[1]    6791 illegal hardware instruction (core dumped)  julia test.jl

@timholy
Copy link
Member

timholy commented Nov 18, 2018

This seems fixed now, OK to close?

@schmrlng
Copy link
Contributor Author

The segfault seems to be fixed but I would still think that f(X) = perp.(X) ./ norm.(X) should be non-allocating when called on a SVector of SVectors. From what I gather, however, this is a base performance issue relating to type inference's recursion limiting heuristic so I agree that this issue can be closed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants