julia 1.11.1: loading non-existing package in multi-process mode causes julia to hang #56345

gconte75 · 2024-10-26T11:14:12Z

On Linux, Debian Buster, Julia 1.11.1 (started as julia -p 4)

Today, I first typed:

using Base.Threads

all good.

then, by mistake, I typed

using Distributef

(instead of using Distributed) instead of rejecting the typo, julia goes on hold for quite a while and then it gives an infinite list of errors related to a number of workers even though I haven't quite done anything.
Sometimes I can try and stop it by pressing ctrl-c or ctrl-x but on more than one occasion I had to kill the terminal window running julia.
Some other times it just crashes.

I have an older version of Julia (1.06 something I believe) on the same Linux machine that correctly rejects the typo with no further issues.

The text was updated successfully, but these errors were encountered:

gconte75 · 2024-10-26T11:24:49Z

It does not happen starting julia with a single ~~thread of execution.~~ process [sentence redacted by @giordano, previous statement was misleading]
I just realized that the -p command line parameter implicitly causes the Distributed package to load into julia, so the using Distributed command is redundant.
Still, it looks like a bug.

giordano · 2024-10-26T15:04:45Z

I can confirm the issue on Julia v1.11.1, but it seems fixed on master:

% julia +nightly -p 2
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.0-DEV.1502 (2024-10-26)
 _/ |\__'_|_|_|\__'_|  |  Commit ee09ae70d9f (0 days old master)
|__/                   |

julia> using DoesNotExist
ERROR: ArgumentError: Package DoesNotExist not found in current path.
- Run `import Pkg; Pkg.add("DoesNotExist")` to install the DoesNotExist package.
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2373 [inlined]
 [2] macro expansion
   @ ./lock.jl:294 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2348
 [4] #invoke_in_world#2
   @ ./essentials.jl:1083 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1080 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2337

The error is thrown almost immediately.

giordano · 2024-10-26T15:13:22Z

For anyone interested in debugging this, the regression seems to have been introduced between v1.11.0-beta2 (good) and v1.11.0-rc1 (bad): v1.11.0-beta2...v1.11.0-rc1

giordano · 2024-10-26T15:21:37Z

This also seems to be "fixed" in #56228 in the sense that an error is thrown relatively quickly, but it involves the infamous REPLExt

$ julia +pr56228 -p 2
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.1 (2024-10-24)
 _/ |\__'_|_|_|\__'_|  |  backports-release-1.11/6be0afc3f76 (fork: 485 commits, 253 days)
|__/                   |

julia> using DoesNotExist
┌ Error: Error during package callback
│   exception =
│    1-element ExceptionStack:
│    On worker 2:
│    ArgumentError: Package REPLExt [e5eb5ef1-03cf-53a7-ae1d-5a66b08e832b] is required but does not seem to be installed:
│     - Run `Pkg.instantiate()` to install all recorded dependencies.
│
│    Stacktrace:
│      [1] _require
│        @ ./loading.jl:2495
│      [2] __require_prelocked
│        @ ./loading.jl:2372
│      [3] #invoke_in_world#3
│        @ ./essentials.jl:1089 [inlined]
│      [4] invoke_in_world
│        @ ./essentials.jl:1086 [inlined]
│      [5] _require_prelocked
│        @ ./loading.jl:2359 [inlined]
│      [6] _require_prelocked
│        @ ./loading.jl:2358 [inlined]
│      [7] macro expansion
│        @ ./lock.jl:273 [inlined]
│      [8] require
│        @ ./loading.jl:2355
│      [9] #2
│        @ /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/Distributed/src/Distributed.jl:83
│     [10] #invokelatest#2
│        @ ./essentials.jl:1055 [inlined]
│     [11] invokelatest
│        @ ./essentials.jl:1052
│     [12] #114
│        @ /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/Distributed/src/process_messages.jl:303
│     [13] run_work_thunk
│        @ /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/Distributed/src/process_messages.jl:70
│     [14] run_work_thunk
│        @ /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/Distributed/src/process_messages.jl:79
│     [15] #100
│        @ /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/Distributed/src/process_messages.jl:88
│
│    ...and 1 more exception.
│
│    Stacktrace:
│      [1] sync_end(c::Channel{Any})
│        @ Base ./task.jl:466
│      [2] macro expansion
│        @ ./task.jl:499 [inlined]
│      [3] _require_callback(mod::Base.PkgId)
│      [3] _require_callback(mod::Base.PkgId)
│        @ Distributed /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/Distributed/src/Distributed.jl:77
│      [4] #invokelatest#2
│        @ ./essentials.jl:1055 [inlined]
│      [5] invokelatest
│        @ ./essentials.jl:1052 [inlined]
│      [6] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1397
│      [7] macro expansion
│        @ ./loading.jl:2656 [inlined]
│      [8] macro expansion
│        @ ./lock.jl:273 [inlined]
│      [9] require_stdlib(package_uuidkey::Base.PkgId, ext::String)
│        @ Base ./loading.jl:2614
│     [10] load_pkg()
│        @ REPL /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/REPL/src/Pkg_beforeload.jl:6
│     [11] check_for_missing_packages_and_run_hooks(ast::Any)
│        @ REPL /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/REPL/src/REPL.jl:268
│     [12] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
│        @ REPL /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/REPL/src/REPL.jl:241
│     [13] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
│        @ REPL /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/REPL/src/REPL.jl:342
│     [14] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
│        @ REPL /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/REPL/src/REPL.jl:327
│     [15] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
│        @ REPL /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/REPL/src/REPL.jl:483
│     [16] run_repl(repl::REPL.AbstractREPL, consumer::Any)
│        @ REPL /data/cceamgi/julia-depot/juliaup/julia-pr56228/share/julia/stdlib/v1.11/REPL/src/REPL.jl:469
│     [17] (::Base.var"#1139#1141"{Bool, Symbol, Bool})(REPL::Module)
│        @ Base ./client.jl:446
│     [18] #invokelatest#2
│        @ ./essentials.jl:1055 [inlined]
│     [19] invokelatest
│        @ ./essentials.jl:1052 [inlined]
│     [20] run_main_repl(interactive::Bool, quiet::Bool, banner::Symbol, history_file::Bool, color_set::Bool)
│        @ Base ./client.jl:430
│     [21] repl_main
│        @ ./client.jl:567 [inlined]
│     [22] _start()
│        @ Base ./client.jl:541
└ @ Base loading.jl:1402
ERROR: ArgumentError: Package DoesNotExist not found in current path.
- Run `import Pkg; Pkg.add("DoesNotExist")` to install the DoesNotExist package.
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2280 [inlined]
 [2] macro expansion
   @ ./lock.jl:273 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2255
 [4] #invoke_in_world#3
   @ ./essentials.jl:1089 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1086 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2248

instead of just the

ArgumentError: Package DoesNotExist not found in current path.

error.

IanButterworth · 2024-10-28T02:04:03Z

I would guess that #56215 being on #56228 is what improved the situation.

IanButterworth · 2024-10-28T02:15:54Z

I can't repro this on #56228 0f7f762

% ./julia --start=no -p 2
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.1 (2024-10-28)
 _/ |\__'_|_|_|\__'_|  |  backports-release-1.11/0f7f762dfe (fork: 488 commits, 254 days)
|__/                   |

julia> using DoesNotExist
ERROR: ArgumentError: Package DoesNotExist not found in current path.
- Run `import Pkg; Pkg.add("DoesNotExist")` to install the DoesNotExist package.
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2280 [inlined]
 [2] macro expansion
   @ ./lock.jl:273 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2255
 [4] #invoke_in_world#3
   @ ./essentials.jl:1089 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1086 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2248

IanButterworth · 2024-10-28T13:04:16Z

@vtjnash is Distributed._require_callback compatible with require_stdlib, given it calls require not require_stdlib on workers?
https://github.com/JuliaLang/Distributed.jl/blob/729ba6aa45a9c3e1d492b53ecb9f6190f05f6428/src/Distributed.jl#L74

I can't say I tested the Pkg repl stuff before this with multiple -p workers.

vtjnash · 2024-10-28T13:27:47Z

I am not sure _require_callback is compatible with anything 😂 but potentially it should not happen from require_stdlib, or at least this callback should decide to check if it was loaded normally

giordano changed the title ~~julia 1.11.1: typo causes hell to break loose~~ julia 1.11.1: loading non-existing package in multi-process mode causes julia to hang Oct 26, 2024

giordano added parallelism Parallel or distributed computation packages Package management and loading bisect wanted labels Oct 26, 2024

giordano added the regression 1.11 Regression in the 1.11 release label Oct 26, 2024

stemann mentioned this issue Nov 5, 2024

Julia 1.11.0-1.11.1 hangs while testing/precompiling a basic Test project with julia_args = ["--threads=auto"] #56458

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

julia 1.11.1: loading non-existing package in multi-process mode causes julia to hang #56345

julia 1.11.1: loading non-existing package in multi-process mode causes julia to hang #56345

gconte75 commented Oct 26, 2024 •

edited by giordano

Loading

gconte75 commented Oct 26, 2024 •

edited by giordano

Loading

giordano commented Oct 26, 2024

giordano commented Oct 26, 2024

giordano commented Oct 26, 2024

IanButterworth commented Oct 28, 2024

IanButterworth commented Oct 28, 2024

IanButterworth commented Oct 28, 2024

vtjnash commented Oct 28, 2024

julia 1.11.1: loading non-existing package in multi-process mode causes julia to hang #56345

julia 1.11.1: loading non-existing package in multi-process mode causes julia to hang #56345

Comments

gconte75 commented Oct 26, 2024 • edited by giordano Loading

gconte75 commented Oct 26, 2024 • edited by giordano Loading

giordano commented Oct 26, 2024

giordano commented Oct 26, 2024

giordano commented Oct 26, 2024

IanButterworth commented Oct 28, 2024

IanButterworth commented Oct 28, 2024

IanButterworth commented Oct 28, 2024

vtjnash commented Oct 28, 2024

gconte75 commented Oct 26, 2024 •

edited by giordano

Loading

gconte75 commented Oct 26, 2024 •

edited by giordano

Loading