timer_callback is causing segfaults with curl 7.81.0 #172

mkitti · 2022-01-08T21:54:48Z

Several unsupported builds of Julia 1.7.1 are reporting segmentation faults when trying to build with curl 7.81.0:

The segmentation fault can be replicated as follows:

$ julia -E 'download("https://curl.se")'

signal (11): Segmentation fault
in expression starting at none:1
prune_dead_connections at /home/mkitti/src/curl/lib/url.c:1092
create_conn at /home/mkitti/src/curl/lib/url.c:3869
Curl_connect at /home/mkitti/src/curl/lib/url.c:4129
multi_runsingle at /home/mkitti/src/curl/lib/multi.c:1860
multi_socket at /home/mkitti/src/curl/lib/multi.c:3131
curl_multi_socket_action at /home/mkitti/src/curl/lib/multi.c:3252
curl_multi_socket_action at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/LibCURL/src/lC_curl_h.jl:230 [inlined]
curl_multi_socket_action at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:91 [inlined]
macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:35 [inlined]
#47 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:147
lock at ./lock.jl:190
timer_callback at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:146
jfptr_timer_callback_19000.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jlcapi_timer_callback_18861.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
Curl_update_timer at /home/mkitti/src/curl/lib/multi.c:3365
curl_multi_add_handle at /home/mkitti/src/curl/lib/multi.c:504
curl_multi_add_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/LibCURL/src/lC_curl_h.jl:194 [inlined]
macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:35 [inlined]
#27 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:51
lock at ./lock.jl:190
add_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:44 [inlined]
#9 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:345
with_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Curl.jl:64
#8 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:311 [inlined]
arg_write at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:112
#7 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:310 [inlined]
arg_read at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:61 [inlined]
#request#5 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:309
request##kw at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:293 [inlined]
#3 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:222 [inlined]
arg_write at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:101
#download#2 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:221 [inlined]
download at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:221
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
jl_f__call_latest at /usr/local/src/conda/julia-1.7.1/src/builtins.c:757
#invokelatest#2 at ./essentials.jl:716 [inlined]
invokelatest at ./essentials.jl:714 [inlined]
do_download at ./download.jl:24 [inlined]
download at ./download.jl:20
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
do_call at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:126
eval_value at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:215
eval_stmt_value at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:166 [inlined]
eval_body at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:601
jl_interpret_toplevel_thunk at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:731
jl_toplevel_eval_flex at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:885
jl_toplevel_eval_flex at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:830
jl_toplevel_eval_in at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:944
eval at ./boot.jl:373 [inlined]
exec_options at ./client.jl:270
_start at ./client.jl:495
jfptr__start_40346.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
true_main at /usr/local/src/conda/julia-1.7.1/src/jlapi.c:559
jl_repl_entrypoint at /usr/local/src/conda/julia-1.7.1/src/jlapi.c:701
main at julia (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x5626d2001098)
Allocations: 2720 (Pool: 2711; Big: 9); GC: 0
Segmentation fault (core dumped)

In this case curl is configured according to
https://github.com/conda-forge/curl-feedstock/blob/master/recipe/build.sh#L8-L17
and is built with openssl:

$ julia -E "using LibCURL; unsafe_string(LibCURL.curl_version())"
"libcurl/7.81.0-DEV OpenSSL/3.0.0 zlib/1.2.11 libssh2/1.10.0 nghttp2/1.43.0"

I bisected the issue to the following commit and line:
curl/curl@2b3dd01#diff-f25cb725c8f9fcc9a6532bed11a8f9c22b60c17edd2d4a3615bd7ceaa6ad8c18R489

The text was updated successfully, but these errors were encountered:

mkitti · 2022-01-08T22:27:18Z

I believe this may have been resolved by #164

giordano · 2022-01-08T22:32:57Z

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

ngam · 2022-01-08T22:34:08Z

And this hasn't been shipped with the 1.7.1 version? What version of Downloads.jl are we supposed to be using?

ngam · 2022-01-08T22:35:55Z

@mkitti Let's make a patch for this and call it a day then? I've been learning towards disabling system libgit2 and mbedtls (to make our linux/osx approach unified)

mkitti · 2022-01-08T22:38:12Z

Julia 1.7.1 uses Downloads.jl 1.5.2:
https://github.com/JuliaLang/julia/blob/v1.7.1/stdlib/Downloads.version

ngam · 2022-01-08T22:38:51Z

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

@mkitti, let's make a dev branch for these types of situations on our end, thoughts? So we can keep testing.

mkitti · 2022-01-08T22:43:07Z

If you want to play Dr. Frankenstein, we could try changing Downloads.version to

DOWNLOADS_BRANCH = release-1.6
DOWNLOADS_SHA1 = 43899e71eed709866f96dd624f1c369a1c8e26a3
DOWNLOADS_GIT_URL := git://github.com/JuliaLang/Downloads.jl.git
DOWNLOADS_TAR_URL = https://api.github.com/repos/JuliaLang/Downloads.jl/tarball/$1

ngam · 2022-01-08T22:44:10Z

If you want to play Dr. Frankenstein, we could try changing Downloads.version to

DOWNLOADS_BRANCH = release-1.6
DOWNLOADS_SHA1 = 43899e71eed709866f96dd624f1c369a1c8e26a3
DOWNLOADS_GIT_URL := git://github.com/JuliaLang/Downloads.jl.git
DOWNLOADS_TAR_URL = https://api.github.com/repos/JuliaLang/Downloads.jl/tarball/$1

LOL NO!!! I don't wanna play anything. I just want this to work 😢

ngam · 2022-01-08T22:48:23Z

the diff: v1.5.2...v1.6.0

mkitti · 2022-01-08T22:53:37Z

Well that 4 line patch will likely make it "work".

mkitti · 2022-01-08T22:59:35Z

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

I built master, and it appears to work with conda-forge's build of curl 7.81.0. I also just did a build of 1.7.1 with Downloads.jl 1.6, and that also appears to work with conda-forge's build of curl 7.81.0.

On the JuliaLang side, I believe this is resolved. However, it may be useful for someone to think about other consequences of curl/curl@2b3dd01 so I'll keep this open until sometime next week.

Feel free to close this though.

ngam · 2022-01-08T23:01:19Z

I believe this may have been resolved by #164

That's already in Julia master, did you try a nightly build?

I built master, and it appears to work with conda-forge's build of curl 7.81.0. I also just did a build of 1.7.1 with Downloads.jl 1.6, and that also appears to work with conda-forge's build of curl 7.81.0.

On the JuliaLang side, I believe this is resolved. However, it may be useful for someone to think about other consequences of curl/curl@2b3dd01 so I'll keep this open until sometime next week.

Feel free to close this though.

Perf!

Let's just stitch downloads 1.6 in there for now

giordano · 2022-01-08T23:09:05Z

Note that you don't need to rebuild Julia, you can follow https://github.com/JuliaLang/Pkg.jl/blob/7af3e27e3784ccde287b4bcb87891a22830a7dee/README.md#using-the-development-version-of-pkgjl to use a different version of Downloads.jl than the one shipped by Julia

ericphanson · 2022-01-09T01:22:32Z

My opinion is that these bugfixes should be backported to Julia 1.7.2 (some discussion here: JuliaLang/julia#43430)

The version bump to curl (Homebrew#92549) is causing segfaults when installing Julia packages. See Homebrew/discussions#2749. Upstream is already aware of the issue (JuliaLang/Downloads.jl#172). I've updated the test to make sure we catch this regression in the future.

The version bump to curl (#92549) is causing segfaults when installing Julia packages. See Homebrew/discussions#2749. Upstream is already aware of the issue (JuliaLang/Downloads.jl#172). I've updated the test to make sure we catch this regression in the future. Closes #92799. Signed-off-by: BrewTestBot <[email protected]>

See: * JuliaLang/Downloads.jl#172 * conda-forge/julia-feedstock#169 (comment) PR: 261417 Reported by: [email protected]

giordano · 2022-03-19T14:28:56Z

so I'll keep this open until sometime next week.

Can this issue be closed now? A few weeks passed since then

mkitti · 2022-03-19T16:54:11Z

We can close this. Any thoughts on @ericphanson 's backport request?

giordano · 2022-03-19T17:00:12Z

Any thoughts on @ericphanson 's backport request?

Already done 2 weeks ago: JuliaLang/julia#44433

mkitti mentioned this issue Jan 8, 2022

Bump (Lib)CURL to 7.81.0 JuliaPackaging/Yggdrasil#4200

Merged

carlocab mentioned this issue Jan 9, 2022

julia: fix segfaults with curl 7.81 Homebrew/homebrew-core#92799

Closed

6 tasks

ericonr mentioned this issue Jan 22, 2022

julia-1.6.1 incompatible with (lib)curl-7.81.0 (seg fault) void-linux/void-packages#35186

Closed

freebsd-git pushed a commit to freebsd/freebsd-ports that referenced this issue Jan 25, 2022

lang/julia: unbreak build when using libcurl 7.81

175c1f5

See: * JuliaLang/Downloads.jl#172 * conda-forge/julia-feedstock#169 (comment) PR: 261417 Reported by: [email protected]

5u623l20 pushed a commit to 5u623l20/freebsd-ports that referenced this issue Jan 27, 2022

lang/julia: unbreak build when using libcurl 7.81

90d2595

See: * JuliaLang/Downloads.jl#172 * conda-forge/julia-feedstock#169 (comment) PR: 261417 Reported by: [email protected]

giordano closed this as completed Mar 19, 2022

cho-m mentioned this issue May 29, 2022

julia 1.7.3 Homebrew/homebrew-core#102342

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

timer_callback is causing segfaults with curl 7.81.0 #172

timer_callback is causing segfaults with curl 7.81.0 #172

mkitti commented Jan 8, 2022

mkitti commented Jan 8, 2022

giordano commented Jan 8, 2022

ngam commented Jan 8, 2022

ngam commented Jan 8, 2022 •

edited

Loading

mkitti commented Jan 8, 2022

ngam commented Jan 8, 2022

mkitti commented Jan 8, 2022

ngam commented Jan 8, 2022

ngam commented Jan 8, 2022 •

edited

Loading

mkitti commented Jan 8, 2022

mkitti commented Jan 8, 2022

ngam commented Jan 8, 2022

giordano commented Jan 8, 2022

ericphanson commented Jan 9, 2022 •

edited

Loading

giordano commented Mar 19, 2022

mkitti commented Mar 19, 2022

giordano commented Mar 19, 2022 •

edited

Loading

timer_callback is causing segfaults with curl 7.81.0 #172

timer_callback is causing segfaults with curl 7.81.0 #172

Comments

mkitti commented Jan 8, 2022

mkitti commented Jan 8, 2022

giordano commented Jan 8, 2022

ngam commented Jan 8, 2022

ngam commented Jan 8, 2022 • edited Loading

mkitti commented Jan 8, 2022

ngam commented Jan 8, 2022

mkitti commented Jan 8, 2022

ngam commented Jan 8, 2022

ngam commented Jan 8, 2022 • edited Loading

mkitti commented Jan 8, 2022

mkitti commented Jan 8, 2022

ngam commented Jan 8, 2022

giordano commented Jan 8, 2022

ericphanson commented Jan 9, 2022 • edited Loading

giordano commented Mar 19, 2022

mkitti commented Mar 19, 2022

giordano commented Mar 19, 2022 • edited Loading

ngam commented Jan 8, 2022 •

edited

Loading

ngam commented Jan 8, 2022 •

edited

Loading

ericphanson commented Jan 9, 2022 •

edited

Loading

giordano commented Mar 19, 2022 •

edited

Loading