Function generator appears to be corrupting the method table

[andyferris]

We have an example of spurious behavior in StaticArrays where calling a @generated function appears to result in a method table being "corrupted" (or otherwise making some methods not available for dispatch).

Full context, including discussions with @vtjnash are available here. A mini-repro thanks to @c42f on latest master is this: (sorry this one isn't correct)

julia> foo(::Type{Int}) = 1
foo (generic function with 1 method)

julia> @generated bar(x) = :($(foo(x)))
bar (generic function with 1 method)

julia> bar(42)
1

julia> foo(::Type{Float64}) = -1
foo (generic function with 2 methods)

julia> bar(42.0)
ERROR: MethodError: no method matching foo(::Type{Float64})
The applicable method may be too new: running in world age 21461, while current world is 21462.
Closest candidates are:
  foo(::Type{Float64}) at REPL[4]:1 (method too new to be called from this world context.)
  foo(::Type{Int64}) at REPL[1]:1
Stacktrace:
 [1] bar(...) at ./REPL[2]:1

julia> foo(1)
ERROR: MethodError: no method matching foo(::Int64)
Closest candidates are:
  foo(::Type{Int64}) at REPL[1]:1
  foo(::Type{Float64}) at REPL[4]:1

julia> methods(foo)
# 2 methods for generic function "foo":
foo(::Type{Int64}) in Main at REPL[1]:1
foo(::Type{Float64}) in Main at REPL[4]:1

julia> foo(1.0)
ERROR: MethodError: no method matching foo(::Float64)
Closest candidates are:
  foo(::Type{Int64}) at REPL[1]:1
  foo(::Type{Float64}) at REPL[4]:1

This is related to the fact that the (pure?) generator is not finding the new method for foo - which I did find surprising at first, but the real problem here is the fact that we can no longer call foo after calling bar.

[yuyichao]

Everything seems correct??

julia> foo(::Type{Int}) = 1
foo (generic function with 1 method)

julia> @generated bar(x) = foo(x)
bar (generic function with 1 method)

julia> bar(42)
1

julia> foo(::Type{Float64}) = -1
foo (generic function with 2 methods)

julia> bar(42.0)
ERROR: fooMethodError: no method matching foo(::Type{Float64})
The applicable method may be too new: running in world age 21451, while current world is 21452.
Closest candidates are:
  foo(::Type{Float64}) at REPL[4]:1 (method too new to be called from this world context.)
  foo(::Type{Int64}) at REPL[1]:1
Stacktrace:
 [1] bar(...) at ./REPL[2]:1

julia> foo(1)
ERROR: MethodError: no method matching foo(::Int64)
Closest candidates are:
  foo(::Type{Int64}) at REPL[1]:1
  foo(::Type{Float64}) at REPL[4]:1

julia> foo(Int)
1

julia> methods(foo)
# 2 methods for generic function "foo":
foo(::Type{Int64}) in Main at REPL[1]:1
foo(::Type{Float64}) in Main at REPL[4]:1

julia> foo(1.0)
ERROR: MethodError: no method matching foo(::Float64)
Closest candidates are:
  foo(::Type{Int64}) at REPL[1]:1
  foo(::Type{Float64}) at REPL[4]:1

julia> foo(Float64)
-1

[andyferris]

Sorry, bad repro!

There is an example in the linked discussion above, JuliaArrays/StaticArrays.jl#106 (comment)

[andyferris]

OK this is a subtle one, depending on the order of function to execute. Both of these are generated by latest master of Julia and StaticArrays:

julia> using StaticArrays

julia> immutable Foo{N, T} <: StaticVector{T}
                     data::NTuple{N, T}
                 end

julia> StaticArrays.Size{N, T}(::Type{Foo{N, T}}) = Size(N)

julia> Base.getindex(f::Foo, i) = f.data[i]

julia> f = Foo(1,2,3);

julia> Size(typeof(f))
Size(3,)

This is identical, except that a semicolon is omitted, and show calls getindex which is a generated function whose generator depends on Size(typeof(f)). The generated function was defined in a "world" before Foo existed and (apparently) doesn't get access to the newly-defined method for Size. The most alarming part is the difference in the final line, however, which worked in the session above.

julia> using StaticArrays

julia> immutable Foo{N, T} <: StaticVector{T}
                     data::NTuple{N, T}
                 end

julia> StaticArrays.Size{N, T}(::Type{Foo{N, T}}) = Size(N)

julia> Base.getindex(f::Foo, i) = f.data[i]

julia> f = Foo(1,2,3)
WARNING: Base.LinearFast is deprecated, use Base.IndexLinear instead.
  likely near no file:0
WARNING: Base.LinearFast is deprecated, use Base.IndexLinear instead.
  likely near no file:0
WARNING: Base.LinearFast is deprecated, use Base.IndexLinear instead.
  likely near no file:0
3-element Foo{3,Int64}:
Error showing value of type Foo{3,Int64}:
ERROR: The size of type `Foo{3,Int64}` is not known.

If you were trying to construct (or `convert` to) a `StaticArray` you
may need to add the size explicitly as a type parameter so its size is
inferrable to the Julia compiler (or performance would be terrible). For
example, you might try

    m = zeros(3,3)
    SMatrix(m)      # this error
    SMatrix{3,3}(m) # correct - size is inferrable

Stacktrace:
 [1] StaticArrays.Size(::Type{Foo{3,Int64}}) at /home/ferris/.julia/v0.6/StaticArrays/src/traits.jl:42
 [2] size(::Type{Foo{3,Int64}}, ::Int64) at /home/ferris/.julia/v0.6/StaticArrays/src/abstractarray.jl:10
 [3] getindex(...) at /home/ferris/.julia/v0.6/StaticArrays/src/indexing.jl:361
 [4] alignment(::IOContext{Base.Terminals.TTYTerminal}, ::Foo{3,Int64}, ::Array{Int64,1}, ::Array{Int64,1}, ::Int64, ::Int64, ::Int64) at ./show.jl:1345
 [5] print_matrix(::IOContext{Base.Terminals.TTYTerminal}, ::Foo{3,Int64}, ::String, ::String, ::String, ::String, ::String, ::String, ::Int64, ::Int64) at ./show.jl:1474
 [6] print_matrix(::IOContext{Base.Terminals.TTYTerminal}, ::Foo{3,Int64}, ::String, ::String, ::String) at ./show.jl:1446
 [7] #showarray#254(::Bool, ::Function, ::IOContext{Base.Terminals.TTYTerminal}, ::Foo{3,Int64}, ::Bool) at ./show.jl:1696
 [8] display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::MIME{Symbol("text/plain")}, ::Foo{3,Int64}) at ./REPL.jl:122
 [9] display(::Base.REPL.REPLDisplay{Base.REPL.LineEditREPL}, ::Foo{3,Int64}) at ./REPL.jl:125
 [10] display(::Foo{3,Int64}) at ./multimedia.jl:194

julia> Size(typeof(f))
ERROR: The size of type `Foo{3,Int64}` is not known.

If you were trying to construct (or `convert` to) a `StaticArray` you
may need to add the size explicitly as a type parameter so its size is
inferrable to the Julia compiler (or performance would be terrible). For
example, you might try

    m = zeros(3,3)
    SMatrix(m)      # this error
    SMatrix{3,3}(m) # correct - size is inferrable

Stacktrace:
 [1] StaticArrays.Size(::Type{Foo{3,Int64}}) at /home/ferris/.julia/v0.6/StaticArrays/src/traits.jl:42

[JeffBezanson]

@timholy has cited this as a blocker, provisionally adding to 0.6 milestone.

[vtjnash]

Is this a bug? The StaticArrays code was annotating methods incorrectly, which resulted in explicit requests from inference to corrupt the method table (due to it assuming that the methods were marked correctly).

[timholy]

I should say that I haven't run into this specific issue, it came up in a search of issues for "world" and so I lumped it with my other world-age issues. The 2nd example in #20703 (comment) seems to work now (if one adds ::Int in the getindex specialization).