Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip building wasm-opt-sys when it has already been built #148

Open
clearloop opened this issue Jul 22, 2023 · 5 comments
Open

Skip building wasm-opt-sys when it has already been built #148

clearloop opened this issue Jul 22, 2023 · 5 comments

Comments

@clearloop
Copy link

the build.rs of wasm-opt-sys takes too much time

@brson
Copy link
Owner

brson commented Jul 26, 2023

The lack of caching in the C++ build step is the big disadvantage of how we're building binaryen using only the cc crate and without cmake. Any time the wasm-opt-sys build script is invoked it will rebuild all the C++ files.

cc explicitly doesn't support any kind of caching or incremental compilation - it's out of scope for that crate.

Fixing this is possible though not trivial.

We could just use cmake but that imposes multiple new external dependencies that are less likely to be available than a C++ compiler, the reason I decided to use cc for the build.

We could also attempt to add a caching layer on top of the cc crate. Even an imperfect implementation would probably be usable for the purpose of not rebuilding all of binaryen. Other projects might want this as well.

My preference is to the latter. It's something I would be interested in doing, but I'll need to see whether it's in scope of the funding for this project.

@brson
Copy link
Owner

brson commented Aug 31, 2023

I've thought a bit about how to add a caching layer on top of the cc crate, and I have not come up with a plan that I think will obviously work. The big obstacle is that we also have to interop with the cxx_build crate, which is itself a wrapper around the cc::Build constructor; and I don't see a way to inject any behavior between cxx_build and cc::Build.

cxx_build is pretty simple though so one possibility is to just copy that code and duplicate its behavior.

@brson
Copy link
Owner

brson commented Sep 14, 2023

I've thought about this some more, and think there may be a way to cache cc build results even with cxx_build, at least with the limited features that wasm-opt uses. I've sketched out an API and docs here:

https://github.com/brson/wasm-opt-rs/blob/cache/components/wasm-opt-sys/cc_cache.rs

I am feeling wary of actually attempting this since there are so many unknown ways caching a C++ build can go wrong. But idk maybe we'll try it.

@brson
Copy link
Owner

brson commented Sep 14, 2023

One thing to note is that even if we skip building some C++ files, I don't see any way to avoid rerunning ar to create the library rustc needs to link to, and the time to run ar is not insignificant.

@brson
Copy link
Owner

brson commented Sep 14, 2023

There is another potential approach here: sscache already does build caching for rust and C++. It could be possible to automatically acquire sccache and configure it as part of the build script.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants