Calls curl_multi_assign with null
#260
Comments
|
I am affected by this issue as well. I can't update any packages i've installed because a segmentation fault at |
|
@bevanjkay do you have a self-contained reproducer? |
|
If I change diff --git a/src/Curl/Curl.jl b/src/Curl/Curl.jl
index fd9b4c9..a2cc146 100644
--- a/src/Curl/Curl.jl
+++ b/src/Curl/Curl.jl
@@ -51,6 +51,7 @@ function curl_multi_socket_action(multi_handle, s, ev_bitmask, running_handles)
ccall((:curl_multi_socket_action, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Cint, Ptr{Cint}), multi_handle, s, ev_bitmask, running_handles)
end
function curl_multi_assign(multi_handle, sockfd, sockp)
+ multi_handle == C_NULL && error("multi_handle is NULL")
ccall((:curl_multi_assign, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Ptr{Cvoid}), multi_handle, sockfd, sockp)
end
all tests pass for me. Please, provide a self-contained minimal reproducer, otherwise it looks complicated to address this issue. |
|
And with diff --git a/src/Curl/Curl.jl b/src/Curl/Curl.jl
index fd9b4c9..546ef21 100644
--- a/src/Curl/Curl.jl
+++ b/src/Curl/Curl.jl
@@ -51,6 +51,8 @@ function curl_multi_socket_action(multi_handle, s, ev_bitmask, running_handles)
ccall((:curl_multi_socket_action, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Cint, Ptr{Cint}), multi_handle, s, ev_bitmask, running_handles)
end
function curl_multi_assign(multi_handle, sockfd, sockp)
+ @info "Hello from curl_multi_assign"
+ multi_handle == C_NULL && error("multi_handle is NULL")
ccall((:curl_multi_assign, libcurl), CURLMcode, (Ptr{CURLM}, curl_socket_t, Ptr{Cvoid}), multi_handle, sockfd, sockp)
end
I tried to follow the (non-minimal) reproducer by @carlocab in curl/curl#14860 (comment) and I still can't see % julia -p 3 --project=/tmp -e 'using Pkg; Pkg.add("Example")'
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
Resolving package versions...
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
[ Info: Hello from curl_multi_assign
Installed Example ─ v0.5.3
Updating `/private/tmp/Project.toml`
[7876af07] + Example v0.5.3
Updating `/private/tmp/Manifest.toml`
[7876af07] + Example v0.5.3
Precompiling packages finished.
1 dependency successfully precompiled in 1 seconds. 25 already precompiled.Note that |
|
Alright, I finally managed to hit a NULL handle with % julia --project=/tmp -e 'using Downloads; Downloads.download("https://example.org", IOBuffer())'
┌ Error: socket_callback: unexpected error
│ err = multi.handle is NULL
└ @ Downloads.Curl ~/repo/Downloads.jl/src/Curl/Multi.jl:204but only with libcurl 8.10.0+. This is happening at the exit of julia, so probably in the finalizer of the Lines 93 to 101 in 89d3c7d All my tests above were with Julia master ( |
|
I don't understand the linking between libcurl, libcurl_jll, etc. very well, so please forgive me if this isn't adding any new information. I hit this same segfault using the Here is the trace: I am running: On a linux docker container running on x86. It's rather difficult to pull more details since this is a non-interactive app running in the cloud, but if there is anything I can provide that would be helpful, I'm happy to get it. |
|
I am also receiving the same thing since openSUSE ships the same libcurl version as well or at least greater than what Julia provides in its bundled deps.
|
|
I can reproduce with |
|
Seems to be resolved with this patch curl/curl@48f61e7. I will test this on an isolated build environment. I should send patches to the curl maintainer of opensuse. EDIT: seems they already pushed a patched version of curl and i didn't see that. just need to wait for image rebuild to happen in the coming weeks :) |
|
I managed to solve this problem by uninstalling the AUR version and installing it the official way according to the instructions on their website ¯\_(ツ)_/¯ |
Welp. Good for you. What I did was use the leap variant of the builds using distrobox. seems it's still not fixed on tumbleweed with curl v8.10.0+ something |
In julia side:
In libcurl side:
patch1. clean and set to nullDownloads.jl/src/Curl/Multi.jl Lines 24 to 31 in 39036e1 - multi.handle = C_NULL
curl_multi_cleanup(handle)
+ multi.handle = C_NULL ref:
2. check null in
|
|
Did you actually try the suggested patch? I seem to remember I tried already - multi.handle = C_NULL
curl_multi_cleanup(handle)
+ multi.handle = C_NULL but that broke something else. |
|
Also, I suggested multi = unsafe_pointer_to_objref(multi_p)::Multi
+ multi.handle == C_NULL && return -1
+
if watcher_p != C_NULLat https://julialang.slack.com/archives/C01439FAFA7/p1726680870906299 but @vtjnash said that would introduce a data race. You can see that in that thread I discussed several solution, including adding a finalizing flag diff --git a/src/Curl/Multi.jl b/src/Curl/Multi.jl
index d2be032..fad991f 100644
--- a/src/Curl/Multi.jl
+++ b/src/Curl/Multi.jl
@@ -4,9 +4,10 @@ mutable struct Multi
timer :: Union{Nothing,Timer}
easies :: Vector{Easy}
grace :: UInt64
+ finalizing :: Bool
function Multi(grace::Integer = typemax(UInt64))
- multi = new(ReentrantLock(), C_NULL, nothing, Easy[], grace)
+ multi = new(ReentrantLock(), C_NULL, nothing, Easy[], grace, false)
finalizer(done!, multi)
@lock MULTIS_LOCK push!(filter!(m -> m.value isa Multi, MULTIS), WeakRef(multi))
return multi
@@ -26,6 +27,7 @@ function done!(multi::Multi)
handle = multi.handle
handle == C_NULL && return
multi.handle = C_NULL
+ multi.finalizing = true
curl_multi_cleanup(handle)
nothing
end
@@ -162,6 +164,7 @@ function socket_callback(
return -1
end
multi = unsafe_pointer_to_objref(multi_p)::Multi
+ multi.finalizing && return 0
if watcher_p != C_NULL
old_watcher = unsafe_pointer_to_objref(watcher_p)::FDWatcher
@check curl_multi_assign(multi.handle, sock, C_NULL)but none of that was accepted. |
Yes, I've tried all those 4 patch with
test with using LibCURL
libcurl_ver = unsafe_string(LibCURL.curl_version())
println("libcurl_ver: $libcurl_ver")
using Downloads
Downloads.download("https://example.org", IOBuffer());
exit()With LibCURL v8.10.1 (included patch 3):
Only apply patch 2, Without patch 4, In summary:
- multi.handle = C_NULL
curl_multi_cleanup(handle)
+ multi.handle = C_NULL |
this restriction may be lifted after JuliaLang/Downloads.jl#260 is solved
* fix ignored python version setting * limit libcurl version to <3.10 this restriction may be lifted after JuliaLang/Downloads.jl#260 is solved
I remembered correctly: tests are completely broken with the change you suggest (and that I had tried already), they hang forever, besides the failure reported in #261. For the record, julia still crashes with signal 6 at exit on curl/curl@9d32724. |
Actually, there's a testset which just takes a while, that's normal. However with curl 8.10.1 test do crash with the change you suggest (which is probably what I was thinking of): But with curl/curl@461ce6c (which didn't exist when I looked into this two months ago) tests finally do finish successfully, besides the failure in #261, at least there are no crashes nor segfaults. So moving multi.handle = C_NULLafter the cleanup is probably a good solution, once we upgrade to whatever is the next version of curl. |
https://github.com/curl/curl/releases/tag/curl-8_11_0 |
|
hi folks, |
When building the latest version of curl at Homebrew, we ran into a segfault when regression testing Julia as a curl dependent.
The segfault issue has been resolved in curl, however it likely points to an issue in the curl multi handler in Julia.
See curl comment for reference;
curl/curl#14860 (comment)
For our CI build;
Complete logs are available here. The referenced error starts here on macOS 14 arm64.
The text was updated successfully, but these errors were encountered: