Path problems(?) with stdlib jlls

[KristofferC]

In latest PkgEval for 1.6 beta (https://github.com/JuliaCI/NanosoldierReports/blob/master/pkgeval/by_hash/b2484a7_vs_ebcc603/report.md) ~10 packages fail when their _jll try to open a library where the library has a dependency on one of the bundled julia libraries.

The packages are:

• ClimateBase v0.10.2 vs. ClimateBase v0.10.2 (successful)
• DFTK v0.2.3 vs. DFTK v0.2.3 (successful)
• DIVAnd v2.6.5 vs. DIVAnd v2.6.5 (successful)
• DataAssim v0.3.2 vs. DataAssim v0.3.2 (successful)
• Enzyme v0.3.0 vs. Enzyme v0.3.0 (successful)
• GEOTRACES v0.1.5 vs. GEOTRACES v0.1.5 (successful)
• NCDatasets v0.11.1 vs. NCDatasets v0.11.1 (successful)
• NetCDF v0.11.0 vs. NetCDF v0.11.0 (successful)
• Phylo v0.4.3 vs. Phylo v0.4.3 (successful)
• ShallowWaters v0.4.1 vs. ShallowWaters v0.4.1 (successful)

The error takes the form of

ERROR: LoadError: InitError: could not load library "/home/pkgeval/.julia/artifacts/877149aedda703fde4a848bffd955df8934ca876/lib/libnetcdf.so"
libmbedtls.so.12: cannot open shared object file: No such file or directory
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:114
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:114
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/KuIwt/src/products/library_generators.jl:61 [inlined]
  [4] __init__()
    @ NetCDF_jll ~/.julia/packages/NetCDF_jll/y27Nj/src/wrappers/x86_64-linux-gnu.jl:14
  [5] _include_from_serialized(path::String, depmods::Vector{Any})

(libmbedtils) or

ERROR: LoadError: LoadError: InitError: could not load library "/home/pkgeval/.julia/artifacts/2657658477fc19883e4ad106aa22c045265f507f/lib/libEnzyme-9.so"
libLLVM-9jl.so: cannot open shared object file: No such file or directory
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:114
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:114
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/KuIwt/src/products/library_generators.jl:61 [inlined]
  [4] __init__()
    @ Enzyme_jll ~/.julia/packages/Enzyme_jll/lmQwz/src/wrappers/x86_64-linux-gnu-cxx11.jl:8

(libLLVM).

[giordano]

If I understand the problem correctly, this is only due to the fact julia v1.6 uses newer versions of some binary libraries which have a different soname compared to v1.5. This effectively means that the a jll package depending on mbedtls can't possibly work with julia v1.5 and v1.6 at the same time. We need to rebuild it against the newer mbedtls, bump the version, and make explicit the requirement of Julia v1.6

[KristofferC]

Thanks, so not really a release blocker then since it just requires package updates.

[giordano]

I think so, yes

[Alexander-Barth]

I am trying to follow @giordano's advice and rebuild NetCDF_jll with newer versions of std lib dependencies:

diff --git a/N/NetCDF/build_tarballs.jl b/N/NetCDF/build_tarballs.jl
index 01255140..f4b833f3 100644
--- a/N/NetCDF/build_tarballs.jl
+++ b/N/NetCDF/build_tarballs.jl
@@ -66,12 +66,12 @@ products = [
 dependencies = [
     Dependency(PackageSpec(name="HDF5_jll", version="1.12.0")),
     Dependency("Zlib_jll"),
-    Dependency("LibCURL_jll", v"7.71.1"),
+    Dependency("LibCURL_jll", v"7.73.0"),
     # The following libraries are dependencies of LibCURL_jll which is now a
     # stdlib, but the stdlib doesn't explicitly list its dependencies
-    Dependency("LibSSH2_jll", v"1.9.0"),
-    Dependency("MbedTLS_jll", v"2.16.8"),
-    Dependency("nghttp2_jll", v"1.40.0"),
+    Dependency("LibSSH2_jll", v"1.9.1"),
+    Dependency("MbedTLS_jll", v"2.24.0"),
+    Dependency("nghttp2_jll", v"1.41.0"),
 ]
 
 # Build the tarballs, and possibly a `build.jl` as well.

But when I rebuild NetCDF_jll I have these warning:

┌ Warning: Dependency MbedTLS_jll does not have an Artifacts.toml at /mnt/data1/abarth/opt/julia-1.6.0-beta1/share/julia/stdlib/v1.6/MbedTLS_jll/Artifacts.toml!
└ @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/HAVQ2/src/Prefix.jl:474
┌ Warning: Dependency LibSSH2_jll does not have an Artifacts.toml at /mnt/data1/abarth/opt/julia-1.6.0-beta1/share/julia/stdlib/v1.6/LibSSH2_jll/Artifacts.toml!
└ @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/HAVQ2/src/Prefix.jl:474
┌ Warning: Dependency nghttp2_jll does not have an Artifacts.toml at /mnt/data1/abarth/opt/julia-1.6.0-beta1/share/julia/stdlib/v1.6/nghttp2_jll/Artifacts.toml!
└ @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/HAVQ2/src/Prefix.jl:474
┌ Warning: Dependency Zlib_jll does not have an Artifacts.toml at /mnt/data1/abarth/opt/julia-1.6.0-beta1/share/julia/stdlib/v1.6/Zlib_jll/Artifacts.toml!
└ @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/HAVQ2/src/Prefix.jl:474
┌ Warning: Dependency LibCURL_jll does not have an Artifacts.toml at /mnt/data1/abarth/opt/julia-1.6.0-beta1/share/julia/stdlib/v1.6/LibCURL_jll/Artifacts.toml!
└ @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/HAVQ2/src/Prefix.jl:474

and finally the error:

[09:52:25] configure: error: curl required for remote access. Install curl or build with --disable-dap.
[09:52:25]  ---> ./configure --prefix=${prefix} --build=${MACHTYPE} --host=${target} --disable-utilities --enable-shared --disable-static
[09:52:25] Previous command exited with 1
[09:52:25] Child Process exited, exit code 1

For some reason, libcurl cannot be found in the build environment.