Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Cache random stuff #118

Merged
merged 9 commits into from
Apr 22, 2024

Conversation

I-Al-Istannen
Copy link
Collaborator

The CI spends quite some time building the cargo project for the initial stages. The directories added in this PR are marked with the CACHE_DIR flag and therefore cargo should be able to deal with them becoming out of date. I just want to see what, if anything, this can save first. There are a few considerations with cache size one has to think about, as we no longer really have a stable hash to base the cache on. Maybe we could build something from the more internal build files, but that is a problem for later. In the meantime it will just bloat the caches, generating a new one every CI run, and then hoping that GitHub ages them out nicely.

@ZuseZ4
Copy link
Member

ZuseZ4 commented Apr 19, 2024

@wsmoses Do you know if the assumption that the oldest caches get cleaned up holds for the OpenStack runner too?

@ZuseZ4 ZuseZ4 requested a review from wsmoses April 19, 2024 21:23
@I-Al-Istannen I-Al-Istannen force-pushed the chore/cache-random-stuff branch from d7b19a3 to 9676faf Compare April 19, 2024 21:37
Copy link
Member

@wsmoses wsmoses left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know but we can find out. LGTM but can you name the cache rust-xyz just in case caches are per device or org

@I-Al-Istannen
Copy link
Collaborator Author

Yea, the names are pretty much random atm. I first wanted to see whether it helps at all. The Cargo.lock file in src/bootstrap also looks mighty tempting as a hash cache key, that might reduce the amount of caches generated. It doesn't quite seem to be exhaustive though (I have a yoke crate locally in my build folder that I can not find in the dependency tree). But maybe close enough, if the automatic rotation doesn't really like lots of caches.

@ZuseZ4 ZuseZ4 force-pushed the chore/cache-random-stuff branch from 9676faf to 3986bf1 Compare April 19, 2024 22:40
@I-Al-Istannen I-Al-Istannen marked this pull request as ready for review April 22, 2024 13:52
@ZuseZ4 ZuseZ4 merged commit c9b28e7 into EnzymeAD:master Apr 22, 2024
9 of 12 checks passed
@ZuseZ4
Copy link
Member

ZuseZ4 commented Apr 22, 2024

Going from 18-30 to 3 minutes is an awesome speedup.
Thanks a lot!

@I-Al-Istannen I-Al-Istannen deleted the chore/cache-random-stuff branch April 23, 2024 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants