Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use Pkg.precompile during loading via Pkg hook #49242

Conversation

IanButterworth
Copy link
Member

@IanButterworth IanButterworth commented Apr 3, 2023

Closes #48259

Previously

julia> using CSV
[ Info: Precompiling CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b]

julia> 

This PR

julia> using CSV
Precompiling CSV
  20 dependencies successfully precompiled in 30 seconds. 1 already precompiled.

julia> 

Note that errors and warnings will be shown, including warnings about precompilation waiting for tasks to finish etc. so this should be quite safe. But a switch could be added to turn it off.

Requires:

Also two minor bug fixes:

@IanButterworth IanButterworth added the compiler:precompilation Precompilation of modules label Apr 3, 2023
Copy link
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are in the imminent process of removing Pkg, so this seems unwise to merge now

@vtjnash
Copy link
Member

vtjnash commented Apr 3, 2023

There was also an issue with Pkg.precompile suppressing warnings and other similar messages, which means this PR would harm usability and development until that fix is merged.

@ericphanson
Copy link
Contributor

ericphanson commented Apr 3, 2023

There was also an issue with Pkg.precompile suppressing warnings and other similar messages, which means this PR would harm usability and development until that fix is merged.

I believe Ian has already addressed this in Pkg.jl. See also his message:

Note that errors and warnings will be shown, including warnings about precompilation waiting for tasks to finish etc. so this should be quite safe. But a switch could be added to turn it off.


We are in the imminent process of removing Pkg, so this seems unwise to merge now

This just adds a hook, so it seems fine? If you load Pkg, it populates the hook, and switches you to parallel precompilation.

@KristofferC
Copy link
Member

We are in the imminent process of removing Pkg, so this seems unwise to merge now

We might want to move the parallel precompile code to Base. It doesn't really have anything to do with the package manager itself. You would want parallel precompile without having to load Pkg.

@vtjnash
Copy link
Member

vtjnash commented Apr 4, 2023

The issue seems to be still open #48371?

@IanButterworth
Copy link
Member Author

Oops. Just closed that

@vtjnash
Copy link
Member

vtjnash commented Apr 4, 2023

Gotcha, yeah that makes more sense.

@IanButterworth
Copy link
Member Author

Should this be merged before Pkg.precompile is moved to Base? It might be helpful to try it out and if it's ok aim to move it over for 1.10?

@KristofferC
Copy link
Member

IMO it's fine to merge it in the current state (using Pkg) so the whole concept gets tested.

@IanButterworth IanButterworth merged commit 4134e93 into JuliaLang:master Apr 5, 2023
@IanButterworth IanButterworth deleted the ib/code_load_parallel_precomp branch April 5, 2023 02:28
@IanButterworth
Copy link
Member Author

Won't be functional until Pkg has been bumped with JuliaLang/Pkg.jl#3433

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:precompilation Precompilation of modules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

using should trigger Pkg's parallel precompilation
4 participants