Increase timeout for bootstrap running. #7449
Conversation
Darwin specifically moved to 15 minutes because that one seems to periodically time out.
|
@andy31415 can we lower/fix the bootstrap time vs upping timeouts to 10 and 15 minutes? This is significantly impacting build times... |
|
Is there a way to bake CIPD into cache or the docker image (cache for Darwin)? |
There is a way to cache a tarball with GitHub but I'm not sure if it will help with CIPD (it's I/O bound process). We should measure though where the time goes, though. Do we end up compiling python wheels or something? CIPD would be instant with a self-hosted runner because we could just keep the checkout bootstrapped (it is optimized for incremental updates). |
|
@mspang I would love to know that, right now - I will echo @andy31415's concern/fix that this takes a long time, I'm constantly re-spinning jobs that timeout here. If we could reduce this to a docker pull (which is mostly 'quick') it would be a huge build time improvement. Note: I've seen standard network timeouts cause this to take 10+ minutes in CI - I suspect network is a huge contributor here.. |
|
Bootstrap time was one of the things I suggested as good candidates for fixit during todays standup. In the end though I believe it is a bigger pain for reviewers rather than individual contributors so its priority did not seem that high. Would much rather fix it for sure. |
|
Merged as a hotfix to make our ToT less red. Totally agree that this is patchwork and we need to fix bootstrap time instead. 10 minutes is already too long. |
|
What issue is tracking this? |
Darwin specifically moved to 15 minutes because that one seems to periodically time out.
Problem
CI timeouts during "bootstrap'. Seems to mainly happen on darwin tests, assuming GH hardware or network connectivity slower for macs.
Change overview
Change timeouts from 5 to 10 minutes (and 15 minutes for darwin)
Testing
Could not manual test, expect CI itself to serve as a test bed.