Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nightly hanging test env precompilation with JULIA_NUM_THREADS > 1 #53572

Closed
IanButterworth opened this issue Mar 3, 2024 · 5 comments · Fixed by JuliaLang/Pkg.jl#3840
Closed

Nightly hanging test env precompilation with JULIA_NUM_THREADS > 1 #53572

IanButterworth opened this issue Mar 3, 2024 · 5 comments · Fixed by JuliaLang/Pkg.jl#3840
Labels
regression Regression in behavior compared to a previous version
Milestone
1.11

Comments

@IanButterworth
Copy link
Member

ProgressMeter.jl precompilation in the Pkg.test environment (which usually takes 1s) is hanging on nightly in CI when JULIA_NUM_THREADS=2 but not when =1.

1.6, 1.10 and 1.11 all succeed. And precompilation in the regular REPL succeeds.

https://github.com/timholy/ProgressMeter.jl/actions/runs/8127876000/job/22213188493?pr=305

I suspect that JuliaLang/Pkg.jl#3792 is the cause, but I'm not sure why (fyi @KristofferC )
@IanButterworth IanButterworth added the regression Regression in behavior compared to a previous version label Mar 3, 2024
@IanButterworth IanButterworth mentioned this issue Mar 4, 2024
Merged
@KristofferC KristofferC added this to the 1.11 milestone Mar 8, 2024
@gbaraldi
Copy link
Member

So the issue is that https://github.com/JuliaLang/Pkg.jl/blob/28bbbd46f3303920fd52e4eaccc53543f575f75b/src/Operations.jl#L2042-L2052
might pass the --threads flag to the precompile which is not recommended because precompilation with threads is not supported and may lead to fun and interesting bugs.

@gbaraldi
Copy link
Member

This gets better with #53697, but i'd still consider what Pkg is doing a bug.

@IanButterworth
Copy link
Member Author

Wasn't that all avoided with #48592 ?

@vtjnash
Copy link
Member

vtjnash commented Mar 11, 2024

It used to, but Pkg now includes code to forcibly bypass that fix

@IanButterworth
Copy link
Member Author

Ah, yeah this regressed in JuliaLang/Pkg.jl#3792 which forces the gen_subprocess_flags flags through the flags kwarg

https://github.com/IanButterworth/Pkg.jl/blob/28bbbd46f3303920fd52e4eaccc53543f575f75b/src/Operations.jl#L2045-L2052

@IanButterworth IanButterworth mentioned this issue Mar 12, 2024
Merged
@moble moble mentioned this issue Mar 12, 2024
Merged
@stemann stemann mentioned this issue Jan 25, 2025
Open
stemann added a commit to stemann/Pkg.jl that referenced this issue Jan 25, 2025
@stemann
Ensure precompilation when testing runs without threads on Julia 1.11
c78fa8d 
Precompilation with threads is not supported: JuliaLang/julia#53572 (comment)
@stemann stemann mentioned this issue Jan 25, 2025
Open
stemann added a commit to stemann/Pkg.jl that referenced this issue Jan 25, 2025
@stemann
Ensure precompilation when testing runs without threads on Julia 1.11
484ab21 
Precompilation with threads is not supported: JuliaLang/julia#53572 (comment)
stemann added a commit to stemann/Pkg.jl that referenced this issue Jan 25, 2025
@stemann
Ensure precompilation when testing runs without threads on Julia 1.11
20ab98e 
Precompilation with threads is not supported: JuliaLang/julia#53572 (comment)
stemann added a commit to stemann/Pkg.jl that referenced this issue Jan 25, 2025
@stemann
Ensure precompilation when testing runs without threads on Julia 1.11
13607ff 
Fixes JuliaLang/julia#56458

Precompilation with threads is not supported: JuliaLang/julia#53572 (comment)

Ensures that tests run with threads, both when calling `julia --project --threads=auto --eval 'using Pkg; Pkg.test()', and when calling `julia --project --eval 'using Pkg; Pkg.test(; julia_args = ["--threads=auto"])'`.
stemann added a commit to stemann/Pkg.jl that referenced this issue Jan 25, 2025
@stemann
Ensure precompilation when testing runs without threads on Julia 1.11
6c64500 
Fixes JuliaLang/julia#56458

Precompilation with threads is not supported: JuliaLang/julia#53572 (comment)

Ensures that tests run with threads, both when calling `julia --project --threads=auto --eval 'using Pkg; Pkg.test()'`, and when calling `julia --project --eval 'using Pkg; Pkg.test(; julia_args = ["--threads=auto"])'`.
@stemann stemann mentioned this issue Feb 1, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
regression Regression in behavior compared to a previous version
Projects
None yet
Milestone
1.11
Development

Successfully merging a pull request may close this issue.

test env precompile: move threads assignment after precompilation IanButterworth/Pkg.jl
4 participants
@vtjnash @KristofferC @IanButterworth @gbaraldi