Inference disabled by a world age issue (0.6.0-rc3) #22355
Comments
|
Images.jl doesn't compile on 0.7, and this case depends on Images. So I can't test this easily. Though the error is possibly related: julia> using ImageCore
julia: /home/tim/src/julia-1.0/src/gf.c:1321: jl_method_instance_add_backedge: Assertion `callee->min_world <= caller->min_world && callee->max_world >= caller->max_world' failed.
signal (6): Aborted
while loading no file, in expression starting on line 0
raise at /build/glibc-9tT8Do/glibc-2.23/signal/../sysdeps/unix/sysv/linux/raise.c:54
abort at /build/glibc-9tT8Do/glibc-2.23/stdlib/abort.c:89
__assert_fail_base at /build/glibc-9tT8Do/glibc-2.23/assert/assert.c:92
__assert_fail at /build/glibc-9tT8Do/glibc-2.23/assert/assert.c:101
jl_method_instance_add_backedge at /home/tim/src/julia-1.0/src/gf.c:1321
jl_insert_backedges at /home/tim/src/julia-1.0/src/dump.c:2233
_jl_restore_incremental at /home/tim/src/julia-1.0/src/dump.c:3286
jl_restore_incremental at /home/tim/src/julia-1.0/src/dump.c:3325
_include_from_serialized at ./loading.jl:157
_require_from_serialized at ./loading.jl:200
unknown function (ip: 0x7f8248bf2d3b)
jl_call_fptr_internal at /home/tim/src/julia-1.0/src/julia_internal.h:350
jl_call_method_internal at /home/tim/src/julia-1.0/src/julia_internal.h:369
jl_apply_generic at /home/tim/src/julia-1.0/src/gf.c:1925
_require_search_from_serialized at ./loading.jl:236
unknown function (ip: 0x7f8248bf366b)
jl_call_fptr_internal at /home/tim/src/julia-1.0/src/julia_internal.h:350
jl_call_method_internal at /home/tim/src/julia-1.0/src/julia_internal.h:369
jl_apply_generic at /home/tim/src/julia-1.0/src/gf.c:1925
_require at ./loading.jl:433
require at ./loading.jl:397
unknown function (ip: 0x7f8248bf5a19)
jl_call_fptr_internal at /home/tim/src/julia-1.0/src/julia_internal.h:350
jl_call_method_internal at /home/tim/src/julia-1.0/src/julia_internal.h:369
jl_apply_generic at /home/tim/src/julia-1.0/src/gf.c:1925
jl_apply at /home/tim/src/julia-1.0/src/julia.h:1440
eval_import_path_ at /home/tim/src/julia-1.0/src/toplevel.c:403
eval_import_path at /home/tim/src/julia-1.0/src/toplevel.c:431
jl_toplevel_eval_flex at /home/tim/src/julia-1.0/src/toplevel.c:498
jl_toplevel_eval at /home/tim/src/julia-1.0/src/toplevel.c:605
jl_toplevel_eval_in at /home/tim/src/julia-1.0/src/builtins.c:496
eval at ./boot.jl:236
unknown function (ip: 0x7f8248a257cd)
jl_call_fptr_internal at /home/tim/src/julia-1.0/src/julia_internal.h:350
jl_call_method_internal at /home/tim/src/julia-1.0/src/julia_internal.h:369
jl_apply_generic at /home/tim/src/julia-1.0/src/gf.c:1925
eval_user_input at ./REPL.jl:66
unknown function (ip: 0x7f8248ae23cd)
jl_call_fptr_internal at /home/tim/src/julia-1.0/src/julia_internal.h:350
jl_call_method_internal at /home/tim/src/julia-1.0/src/julia_internal.h:369
jl_apply_generic at /home/tim/src/julia-1.0/src/gf.c:1925
macro expansion at ./REPL.jl:97 [inlined]
#1 at ./event.jl:73
unknown function (ip: 0x7f823677261f)
jl_call_fptr_internal at /home/tim/src/julia-1.0/src/julia_internal.h:350
jl_call_method_internal at /home/tim/src/julia-1.0/src/julia_internal.h:369
jl_apply_generic at /home/tim/src/julia-1.0/src/gf.c:1925
jl_apply at /home/tim/src/julia-1.0/src/julia.h:1440
start_task at /home/tim/src/julia-1.0/src/task.c:267
unknown function (ip: 0xffffffffffffffff)
Allocations: 4160249 (Pool: 4158989; Big: 1260); GC: 7
Aborted (core dumped) |
|
I've seen something similar for PyCall. I've just reduced it to __precompile__()
module PyCall
const b = Ref(true)
function __init__()
ntuple(3) do i
b[] ? 1 : ""
end
end
endThis does not directly trigger assertion for me but as shown by The serialized world age range is invalid (min > max), causing the precompiled |
|
Those age ranges aren't invalid, although #22340 may improve that case. If not, you can turn on the |
|
The issue I post above is not solved by #22340
|
The only place this invalid combination can be created is in dump.c, when it is trying to delete an invalid method. Apparently it's not doing a good enough job. |
|
So I'm not sure how the serializer need to handle world ranges but the issue I'm seeing seems to be caused by The following patch seems to solve the issue for the reduced PyCall case above diff --git a/src/dump.c b/src/dump.c
index 8b39646fc8..1f0c1e887d 100644
--- a/src/dump.c
+++ b/src/dump.c
@@ -2180,7 +2180,7 @@ static size_t lowerbound_dependent_world_set(size_t world, arraylist_t *dependen
if (depworld <= world)
return depworld;
}
- return jl_main_module->primary_world;
+ return world;
}
void jl_method_instance_delete(jl_method_instance_t *mi);Unclear if it is the right solution though. |
|
I just ran into the same issue upon
Beginning of traces |
|
Worth pointing out that this is not fixed by #22340 (I don't think it was intended to fix this, but I thought it was worth testing). |
|
I can't reproduce. Can you apply this patch and let us know what it prints? diff --git a/base/inference.jl b/base/inference.jl
index 6cfb076..79dc7bd 100644
--- a/base/inference.jl
+++ b/base/inference.jl
@@ -2357,14 +2357,23 @@ function converge_valid_age!(sv::InferenceState)
end
# work towards converging the valid age range for sv
-function update_valid_age!(min_valid::UInt, max_valid::UInt, sv::InferenceState)
+function update_valid_age!(min_valid::UInt, max_valid::UInt, sv::InferenceState, callee=nothing)
+ minv, maxv = sv.min_valid, sv.max_valid
sv.min_valid = max(sv.min_valid, min_valid)
sv.max_valid = min(sv.max_valid, max_valid)
+ if !(!isdefined(sv.linfo, :def) || !sv.cached || sv.min_valid <= sv.params.world <= sv.max_valid)
+ println("caller: ", sv.linfo)
+ println("callee: ", callee)
+ println("cached: ", sv.cached)
+ println("bounds: ", sv.min_valid, ", ", sv.params.world, ", ", sv.max_valid)
+ println("original bounds: ", minv, ", ", maxv)
+ println("input bounds: ", min_valid, ", ", max_valid)
+ #print_callstack(sv)
+ end
@assert !isdefined(sv.linfo, :def) || !sv.cached || sv.min_valid <= sv.params.world <= sv.max_valid "invalid age range update"
nothing
end
-update_valid_age!(edge::InferenceState, sv::InferenceState) = update_valid_age!(edge.min_valid, edge.max_valid, sv)
-update_valid_age!(li::MethodInstance, sv::InferenceState) = update_valid_age!(min_world(li), max_world(li), sv)
+update_valid_age!(edge::InferenceState, sv::InferenceState) = update_valid_age!(edge.min_valid, edge.max_valid, sv, edge.linfo)
+update_valid_age!(li::MethodInstance, sv::InferenceState) = update_valid_age!(min_world(li), max_world(li), sv, li)
# temporarily accumulate our edges to later add as backedges in the callee
function add_backedge(li::MethodInstance, caller::InferenceState) |
|
@vtjnash, if you want access to our private repos, I can send you an invite. |
|
I keep getting this error on Julia v0.6 in DynamicPolynomials, e.g. here and here. However I cannot create a minimal example since whenever I try to remove lines that are unrelated it stop failing. I can send a zip with MultivariatePolynomials and DynamicPolynomials in a failing state if it helps. |
|
Does the code work on master? |
|
I have just tested it, it works on 472353c. |
|
Would it help to bisect or do you have an idea of the change in Julia 0.7 that might have fixed this ? |
|
A bisect would be helpful I think. I tried checking out a few different branches of these packages and couldn't reproduce. |
|
I have created the packages A.jl and B.jl that are failing (it is just MultivariatePolynomials and DynamicPolynomials renamed in a failing state). Version 0.6.0 (2017-06-19 13:05 UTC)
x86_64-pc-linux-gnu
julia> Pkg.clone("https://github.com/blegat/A.jl.git")
julia> Pkg.clone("https://github.com/blegat/B.jl.git")
julia> using B
INFO: Precompiling module B.
WARNING: An error occurred during inference. Type inference is now partially disabled.
Base.MethodError(f=typeof(Core.Inference.convert)(), args=(Base.AssertionError, "invalid age range update"), world=0x0000000000000ac2)
unknown function (ip: 0x7fc123746d57)
... |
|
I was able to reproduce with that, but bisecting is tricky since there's a big range of the history where |
|
Applying the patch from #22272 appears to address the freeze, so would be great to reduce these as far as possible to use as a test for that. Bisecting with that patch applied earlier in the history |
|
I'm still seeing this issue even with the latest 0.7. I've applied @vtjnash's patch and got: julia> using X
INFO: Recompiling stale cache file /Users/omus/.julia/lib/v0.7/PyCall.ji for module PyCall.
caller: Base.isequal(typeof(Base.isequal), Any, Symbol)
callee: PyCall.==(typeof(Base.:(==)), PyCall.PyObject, Symbol)
cached: true
bounds: 0x0000000000005985, 0x0000000000005985, 0x0000000000000000
original bounds: 0x0000000000005985, 0xffffffffffffffff
input bounds: 0x0000000000000001, 0x0000000000000000
Internal error: encountered unexpected error in runtime:
Base.MethodError(f=typeof(Core.Inference.convert)(), args=(Base.AssertionError, "invalid age range update"), world=0x0000000000000b02)
rec_backtrace at /Users/omus/Development/Julia/latest/src/stackwalk.c:86
record_backtrace at /Users/omus/Development/Julia/latest/src/task.c:246 [inlined]
jl_throw at /Users/omus/Development/Julia/latest/src/task.c:568
jl_method_error_bare at /Users/omus/Development/Julia/latest/src/gf.c:1474
jl_method_error at /Users/omus/Development/Julia/latest/src/gf.c:1492
jl_lookup_generic_ at /Users/omus/Development/Julia/latest/src/gf.c:1906
jl_apply_generic at /Users/omus/Development/Julia/latest/src/gf.c:1926
Type at ./coreimg.jl:16
update_valid_age! at ./inference.jl:2818
add_backedge! at ./inference.jl:2831
...Unfortunately, the package I'm using which produces this problem isn't open source. I'll try to investigate. |
|
That's a different issue triggered by PyPlot in precompilation (it triggers an assertion with julia-debug) Bisected to 7ec9b18. Haven't had time to reduce it yet. |
|
@yuyichao Do you mean |
|
No |
|
In Curtis' example, |
|
So? |
How could |
|
Because the issue is clearly PyCall related even though loading the package itself doesn't. It also shows the same error in type inference. |
|
So you do mean |
|
No I mean Not sure what you are trying to argue. |
|
Ohhh I see. To me, it sounded like you were saying that something in Not trying to argue, just trying to understand what you were saying :) |
|
Has this been backported in to Julia v0.6.1 ? |
|
@ararslan I assume you remove the |
|
I have just tested the Julia v0.6.1 64 bits linux binaries, the bug is indeed still there 😢 |
|
@ViralBShah Correct. If the label is there, it's still pending. |
|
Is this still an issue or is this moot now? |
|
I rewrote these code paths pretty significantly in #31191, so I'd say moot. |
I have a quite complicated example (involving several official packages and a couple of private ones) that I haven't yet succeeded in simplifying, but I'm wondering whether this debug info is enough for a clue. Using this diff against 0.6.0-rc3:
I got this backtrace (during precompilation):
So the problem seems to be that the valid range of world ages for
BufferedStreams.flushbuffer!is empty. (Either something is uninitialized, or a previous error is being encoded this way?)If someone wants to debug this but needs access to my private repos, that can be arranged.
The text was updated successfully, but these errors were encountered: