Better guidance on how to set up CI with Pants. #104
Conversation
|
OK, linked to that page, but also radically changed the caching setup, and will change the corresponding docs in Pants. It's really important to note that GHA will not update its cache if there was a cache hit, so using an underspecified key is actually pretty useless. |
|
PTAL since this is radically different now (and the docs change is too) |
There was a problem hiding this comment.
So it might not be the best beginner-workflow, but I've thought a lot about GitHub Actions caching re: immutable caches.
The best solution I've come up with that balances restore-keys with the global cache limit is to include a "reset" somewhere in the key that'll allow you to cache miss after some criteria. You'd have an action that takes longer to run, but would then result in only fresh entries. An example might be encoding the week since epoch time.
Anywho, may/may not wanna try it out, and may/may not wanna suggest it.
|
If the point of this is to provide general guidance on what is cacheable, I don't feel comfortable including the restore-keys without letting the user know the dangers, because people WILL copy and paste this. Other than that, I think it's an improvement. |
That is an issue with most uses of restore keys, not just Pants. I can add a caveat though. |
|
Added caveats on restore keys, and a simpler way of plucking the version out of pants.toml. |
|
Also, the docs will now link to this instead of replicating it. |
|
I'm always vaguely amazed when handwritten TOML parses on the first go |
|
Yeah, some standard GHA actions that you can plug and go would be great! |
|
@thejcannon wdyt now? |
| id: pants_version | ||
| run: | | ||
| # Capture the "pants_version = " line from config. | ||
| PANTS_VERSION=$(grep -E '^pants_version\s*=' pants.toml) |
There was a problem hiding this comment.
Grep is so much easier, lol 😅
There was a problem hiding this comment.
It's less accurate in that we're pulling out the entire line, and not just the version string. But for invalidation purposes that is fine! As long as no one tries to use that string for other purposes.
This adjusts the cache keys used in the CI setup based on the improved version from #104. I think there were some typos (e.g. using `pants-setup-...` for the `named_caches` restore key), and it seems that scoping everything under `pants-{name}-...` is nifty.
| with: | ||
| path: | | ||
| ~/.cache/pants/lmdb_store | ||
| key: pants-setup-${{ runner.os }}-${{ hashFiles('**/*') }} |
There was a problem hiding this comment.
Doesn't this mean that you will essentially never hit this cache, unless a PR doesn't change any files?
There was a problem hiding this comment.
Yep, exactly. Which is why it's not practical for repos above a certain size. The advantage of it is that with the restore keys you would get some other "close" cache value (the most recent one with the pants-setup-${{ runner.os }}- prefix). I can see this possibly being useful in some scenarios, but probably not generally.
Links back to the docs and explains why remote caching can be helpful.