build requires network access due to Pkg #43006

vtjnash · 2021-11-08T22:15:38Z

Printing out the generate_precompile.jl script, we see that it may waste a lot of time on network access, which it is not supposed to do. With networking partially blocked, I see reported that:

...
Generating REPL precompile statements... 28/35
#### inputting statement: ####
"Pkg.update()"
####
julia> Pkg.update()
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = Operation timed out after 30016 milliseconds with 0 out of 0 bytes received while requesting https://pkg.julialang.org/registries
...

Refs #40185, #41019,

The text was updated successfully, but these errors were encountered:

KristofferC · 2021-11-10T10:18:33Z

Fixed for 1.7. Probably still problematic on master.

vtjnash · 2022-08-10T16:07:59Z

Adding milestone, since this is apparently still "problematic for master". Please close if it is not

KristofferC · 2022-08-11T13:44:44Z

This is an issue on 1.8 but is IMO not release blocking since the worst thing it does is make the build a bit longer.

ViralBShah · 2023-07-19T03:37:45Z

Please reopen if still an issue.

vtjnash · 2023-09-06T12:07:18Z

This seems to be causing problems with moving Pkg out of the sysimg, since it appears to cause that process to crash if there are network issues: #51189

KristofferC · 2023-09-06T12:08:08Z

This seems to be causing problems with moving Pkg out of the sysimg, since it appears to cause that process to crash if there are network issues:

This is not correct since the PR there does not run a precompile workload.

JeroenOlieslagers · 2024-02-15T15:53:42Z

Still an issue on most recent Julia release. Not an issue on 1.9.4 release. When trying to add JLD2 (or any other package), this error presents:

  Installing known registries into `/scratch/jo2229/.julia`
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = RequestError: Error reading ca cert file /etc/ssl/certs/ca-certificates.crt - mbedTLS: (-0x3E00) PK - Read/write of file failed while requesting https://pkg.julialang.org/registries
└ @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:69
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = RequestError: Error reading ca cert file /etc/ssl/certs/ca-certificates.crt - mbedTLS: (-0x3E00) PK - Read/write of file failed while requesting https://pkg.julialang.org/registries
└ @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:69
     Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: LoadError: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:SSL, failed to load CA certificates: 0xffffc200 - PK - Read/write of file failed)

Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Types.jl:70
  [2] clone(io::IOStream, url::String, source_path::String; header::String, credentials::Nothing, kwargs::@Kwargs{})
    @ Pkg.GitTools ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/GitTools.jl:132
  [3] (::Base.var"#96#98"{Base.var"#96#97#99"{Vector{Float64}, Pkg.Registry.var"#51#55", typeof(Pkg.GitTools.clone)}})(::IOStream, ::Vararg{Any}; kwargs::@Kwargs{header::String})
    @ Base ./error.jl:296
  [4] #96
    @ ./error.jl:291 [inlined]
  [5] (::Pkg.Registry.var"#50#54"{Pkg.Registry.RegistrySpec, Nothing, IOStream, String})(tmp::String)
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:238
  [6] mktempdir(fn::Pkg.Registry.var"#50#54"{Pkg.Registry.RegistrySpec, Nothing, IOStream, String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:766
  [7] mktempdir(fn::Function, parent::String)
    @ Base.Filesystem ./file.jl:762
  [8] mktempdir
    @ ./file.jl:762 [inlined]
  [9] (::Pkg.Registry.var"#48#52"{IOStream, Vector{Pkg.Registry.RegistrySpec}, String})()
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:202
 [10] mkpidlock(f::Pkg.Registry.var"#48#52"{IOStream, Vector{Pkg.Registry.RegistrySpec}, String}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
 [11] mkpidlock
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:90 [inlined]
 [12] mkpidlock
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
 [13] download_registries(io::IOStream, regs::Vector{Pkg.Registry.RegistrySpec}, depot::String)
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:168
 [14] download_default_registries(io::IOStream; only_if_empty::Bool, depot::String)
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:111
 [15] download_default_registries
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:98 [inlined]
 [16] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOStream, kwargs::@Kwargs{})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:149
 [17] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:148
 [18] add
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:147 [inlined]
 [19] add(pkg::String)
    @ Pkg.API ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:146
 [20] top-level scope
    @ /scratch/jo2229/state_encoding.jl:2
in expression starting at /scratch/jo2229/state_encoding.jl:2

caused by: GitError(Code:ERROR, Class:SSL, failed to load CA certificates: 0xffffc200 - PK - Read/write of file failed)
Stacktrace:
  [1] set_ssl_cert_locations(cert_loc::String)
    @ LibGit2 ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/LibGit2.jl:1014
  [2] initialize()
    @ LibGit2 ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/LibGit2.jl:991
  [3] (::LibGit2.var"#161#162")()
    @ LibGit2 ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/LibGit2.jl:977
  [4] lock(f::LibGit2.var"#161#162", l::ReentrantLock)
    @ Base ./lock.jl:229
  [5] ensure_initialized
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/LibGit2.jl:973 [inlined]
  [6] LibGit2.GitConfig(level::LibGit2.Consts.GIT_CONFIG)
    @ LibGit2 ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/config.jl:50
  [7] GitConfig
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/config.jl:50 [inlined]
  [8] CredentialPayload
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/types.jl:1367 [inlined]
  [9] #CredentialPayload#23
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/types.jl:1385 [inlined]
 [10] CredentialPayload
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/types.jl:1384 [inlined]
 [11] clone(repo_url::SubString{String}, repo_path::String; branch::String, isbare::Bool, remote_cb::Ptr{Nothing}, credentials::LibGit2.CachedCredentials, callbacks::Dict{Symbol, Tuple{Ptr{Nothing}, Any}})
    @ LibGit2 ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/LibGit2.jl:564
 [12] clone
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/LibGit2/src/LibGit2.jl:558 [inlined]
 [13] clone(io::IOStream, url::String, source_path::String; header::String, credentials::Nothing, kwargs::@Kwargs{})
    @ Pkg.GitTools ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/GitTools.jl:121
 [14] (::Base.var"#96#98"{Base.var"#96#97#99"{Vector{Float64}, Pkg.Registry.var"#51#55", typeof(Pkg.GitTools.clone)}})(::IOStream, ::Vararg{Any}; kwargs::@Kwargs{header::String})
    @ Base ./error.jl:296
 [15] #96
    @ ./error.jl:291 [inlined]
 [16] (::Pkg.Registry.var"#50#54"{Pkg.Registry.RegistrySpec, Nothing, IOStream, String})(tmp::String)
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:238
 [17] mktempdir(fn::Pkg.Registry.var"#50#54"{Pkg.Registry.RegistrySpec, Nothing, IOStream, String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:766
 [18] mktempdir(fn::Function, parent::String)
    @ Base.Filesystem ./file.jl:762
 [19] mktempdir
    @ ./file.jl:762 [inlined]
 [20] (::Pkg.Registry.var"#48#52"{IOStream, Vector{Pkg.Registry.RegistrySpec}, String})()
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:202
 [21] mkpidlock(f::Pkg.Registry.var"#48#52"{IOStream, Vector{Pkg.Registry.RegistrySpec}, String}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
 [22] mkpidlock
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:90 [inlined]
 [23] mkpidlock
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
 [24] download_registries(io::IOStream, regs::Vector{Pkg.Registry.RegistrySpec}, depot::String)
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:168
 [25] download_default_registries(io::IOStream; only_if_empty::Bool, depot::String)
    @ Pkg.Registry ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:111
 [26] download_default_registries
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:98 [inlined]
 [27] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOStream, kwargs::@Kwargs{})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:149
 [28] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:148
 [29] add
    @ ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:147 [inlined]
 [30] add(pkg::String)
    @ Pkg.API ~/.julia/juliaup/julia-1.10.1+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:146
 [31] top-level scope
    @ /scratch/jo2229/state_encoding.jl:2

KristofferC · 2024-02-15T16:04:02Z

When trying to add JLD2 (or any other package), this error presents:

This is #53339. See the thread for a work around in there.

vtjnash added the regression Regression in behavior compared to a previous version label Nov 8, 2021

vtjnash added this to the 1.7 milestone Nov 8, 2021

DilumAluthge added the building Build system, or building Julia or its dependencies label Nov 8, 2021

IanButterworth mentioned this issue Nov 9, 2021

Extend Pkg.offline to optionally handle offline registries. Use offline mode during precompile_script JuliaLang/Pkg.jl#2833

Open

KristofferC mentioned this issue Nov 9, 2021

prevent internet access during precompile script JuliaLang/Pkg.jl#2834

Merged

KristofferC closed this as completed in JuliaLang/Pkg.jl#2834 Nov 9, 2021

KristofferC reopened this Nov 10, 2021

KristofferC removed this from the 1.7 milestone Nov 10, 2021

vtjnash added this to the 1.8 milestone Aug 10, 2022

vtjnash mentioned this issue Aug 31, 2022

julia 1.8.0 hangs on ubuntu_aarch64 during compilation #46431

Closed

mkitti mentioned this issue Sep 2, 2022

try using git system on macos conda-forge/julia-feedstock#223

Draft

1 task

ViralBShah closed this as completed Jul 19, 2023

vtjnash reopened this Sep 6, 2023

vtjnash closed this as completed Feb 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build requires network access due to Pkg #43006

build requires network access due to Pkg #43006

vtjnash commented Nov 8, 2021

KristofferC commented Nov 10, 2021

vtjnash commented Aug 10, 2022

KristofferC commented Aug 11, 2022

ViralBShah commented Jul 19, 2023

vtjnash commented Sep 6, 2023

KristofferC commented Sep 6, 2023

JeroenOlieslagers commented Feb 15, 2024

KristofferC commented Feb 15, 2024

build requires network access due to Pkg #43006

build requires network access due to Pkg #43006

Comments

vtjnash commented Nov 8, 2021

KristofferC commented Nov 10, 2021

vtjnash commented Aug 10, 2022

KristofferC commented Aug 11, 2022

ViralBShah commented Jul 19, 2023

vtjnash commented Sep 6, 2023

KristofferC commented Sep 6, 2023

JeroenOlieslagers commented Feb 15, 2024

KristofferC commented Feb 15, 2024