make sure that the read task is done when precompiling

[KristofferC]

Should fix #29016.

Would be good if you could try this @iblis17.

https://github.com/JuliaLang/julia/pull/29024/files?w=1

[iblislin]

Simple IO redirection is fine. 👍
I'm testing this patch with other buildtools.

[iblislin]

well, I have trouble in backporting it to the v1.0.0 commit.
Seems the file generate_precompile.jl changed a lots.

[KristofferC]

Just using the one from this PR should be fine.

[iblislin]

The new generate_precompile.jl is fine on FreeBSD's poudriere.

[KristofferC]

generate_precompile.py

😨

[iblislin]

haha, I feel sleepy.

[KristofferC]

@iblis17 can you try again please, there was a flaw with the previous commit

[iblislin]

I copy it into the v1.0.0 commit, and run gmake > test.log then the build just hang.... about 5 hrs.

[vchuravy]

When the process started by generated_precompile.jl dies (for whatever reason), the process waiting on its output is blocked. To reproduce it should be enough to just kill that process.

[KristofferC]

Hm, make > test.log works for me locally at least so it is an improvement...

Can you try the branch at https://github.com/JuliaLang/julia/tree/kc/debug_precompile. It contains some debug output so we can see what the REPL does and if it chokes somewhere along the way.

[iblislin]

test case	with IO redirection	OS	status	comment
directly build the branch kc/debug_precompile	✅	FreeBSD	✔️
directly build the branch kc/debug_precompile	✅	Linux	✔️
build with v1.0.0	❌	FreeBSD	❌ (hang)	also, see the instruction in [1]
build with v1.0.0	✅	FreeBSD	❌ (hang)
build with v1.0.0	❌	Linux	❌ (hang)
build with v1.0.0	✅	Linux	❌ (hang)

[1] Here are my instructions about building it with v1.0.0

git checkout kc/debug_precompile
cp contrib/generate_precompile.jl /tmp/tmp.jl
git checkout v1.0.0
cp /tmp/tmp.jl contrib/generate_precompile.jl
(g)make -j 4  # or (g)make -j 4 > /tmp/test.log

[iblislin]

In case of hang, I only can see this

Stdlibs: ────  84.827660 seconds 74.9028%
    JULIA usr/lib/julia/sys-o.a
Generating precompile statements...

[iblislin]

SIGINFO told me that v1.0.0 was choked by readuntil

signal (29): Information request
in expression starting at /data/iblis/git/julia/contrib/generate_precompile.jl:4                                                        
_kevent at /lib/libc.so.7 (unknown line)                                                                                                
_pthread_suspend_all_np at /lib/libthr.so.3 (unknown line)                                                                              
uv__io_poll at /data/iblis/git/julia/deps/srccache/libuv-ed3700c849289ed01fe04273a7bf865340b2bd7e/src/unix/kqueue.c:161                 
uv_run at /data/iblis/git/julia/deps/srccache/libuv-ed3700c849289ed01fe04273a7bf865340b2bd7e/src/unix/core.c:360                        
process_events at ./libuv.jl:98 [inlined]
wait at ./event.jl:246
wait at ./event.jl:46
wait_readnb at ./stream.jl:301
eof at ./stream.jl:61 [inlined]
readuntil_vector! at ./io.jl:689
#readuntil#276 at ./io.jl:767 [inlined]
#readuntil at ./none:0 [inlined]
#readuntil#275 at ./io.jl:762
unknown function (ip: 0x81868b58e)
jl_fptr_trampoline at /data/iblis/git/julia/src/gf.c:1829
#readuntil at ./none:0 [inlined]
#3 at /data/iblis/git/julia/contrib/generate_precompile.jl:95
withenv at ./env.jl:148
jl_fptr_trampoline at /data/iblis/git/julia/src/gf.c:1829
#2 at /data/iblis/git/julia/contrib/generate_precompile.jl:87
mktemp at ./file.jl:555
mktemp at ./file.jl:553 [inlined]
generate_precompile_statements at /data/iblis/git/julia/contrib/generate_precompile.jl:73

[iblislin]

I figured out an extra patch to get it work with v1.0.0

diff --git a/contrib/generate_precompile.jl b/contrib/generate_precompile.jl
index 0bbb3b8065..11e9de09ae 100644
--- a/contrib/generate_precompile.jl
+++ b/contrib/generate_precompile.jl
@@ -88,9 +88,7 @@ function generate_precompile_statements()
                 "TERM" => "") do
             if have_repl
                 p = run(`$(julia_cmd()) -O0 --trace-compile=$precompile_file --sysimage $sysimg
-                        --compile=all --startup-file=no --color=yes
-                        -e 'import REPL; REPL.Terminals.is_precompiling[] = true'
-                        -i`,
+                        --compile=all --startup-file=no --color=yes`,
                         slave, slave, slave; wait=false)
                 readuntil(master, "julia>", keep=true)
                 t = @async begin

[KristofferC]

Alright, I get it now and everything is fine :).

The reason it doesn't work with copying the file to 1.0 is because the change at #28608 is needed, as well.

So cherry picking the commit from that PR and then this one should be enough to get things working.