PVF: consider building workers on-demand in tests

[mrcnski]

Would still like that get_and_check_worker_paths just calls cargo build to provide some better UX for tests. However, this isn't a real blocker as the current behavior isn't changed.

Originally posted by @bkchr in #1771 (review)

See also #1771 (comment).

[eagr]

Wouldn't this cause the tests to time out? Considering the amount of time required to build the workers.

[mrcnski]

I don't think we have timeouts for tests, at least when I do cargo test locally. But yeah, I'm concerned about getting the UX right, especially on poor hardware. My VPS takes ages to build stuff. And I've noticed the situation after #1771 is a bit better so TBH this is a low priority for me. Thanks for the feedback!

[eagr]

I don't think we have timeouts for tests

Ok, I don't know where did I pick that up :)

My VPS takes ages to build stuff

Same here, which got me thinking about getting Apple M2. Esp. the runtime build, which is insane. I also have a build cache that's growing indefinitely. Is it just me? How do you guys cope with that?

[mrcnski]

No, it's not just you. :) We all complain about it. I can confirm that M2 helps a lot - it's crazy how fast it compiles. For runtime builds, they are usually unneeded so you can pass SKIP_WASM_BUILD=1. For the build cache, cargo sweep helps. Here are my notes on it:

-d : dry run
-r : recursive
-i : Keep only artifacts made by Toolchains currently installed by rustup
-t : Number of days backwards to keep

cargo sweep -d -r -i
cargo sweep -r -t 5

I think it misses incremental folders so you have to also fd -I incremental -x rm -R. You may also have to rm -Rf ~/.cargo/{git,registry}.

Or you can just use dust (another Rust utility) to find the biggest folder and cargo clean, it's what I do.

[eagr]

Thanks! Life saver! Some of these should go into the onboarding documentation, if any :))

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/hardware-requirements-for-substrate-engineer/1686/13