Skip to content

Commit

Permalink
[CI] cache yarn instead of node_modules (#25834)
Browse files Browse the repository at this point in the history
The current caching of steps of `node_modules` doesn't work reliable as
is because it includes `arch` in the cache key. `arch` might be
different across workers in the same commit.

I couldn't find a way to optionally restore caches, so what this PR does
is:
- remove the setup step that ran before all other steps and essentially
just populates a circle CI cache key
- all other steps now do: restore yarn cache, `yarn install`, save yarn
cache (fast if already exists)

With this change the initial batch of jobs all race to populate the
cache, but any subsequent jobs should find the existing cache. The
expected downside would be slightly more worker CPU time with all the
parallel jobs, but wall time might be shorter (1 step less in the
critical path) and we should be more reliable as we no longer have the
failure with multiple archs.


## Alternative 1
Remove the `{arch}` from the cache key.

Downside: this might run into weird issues with native dependencies.

## Alternative 2
Somehow check if the cache was restored and only then run a yarn
install.

Downside: couldn't figure out if it's possible to only restore the yarn
cache if restoring the node_modules cache failed. Without that we'd
either always restore both the yarn and node_modules cache or do yarn
installs w/o cache which are prone to failure in the past.
  • Loading branch information
kassens authored Feb 6, 2023
1 parent 03a2160 commit 1445acf
Showing 1 changed file with 67 additions and 128 deletions.
Loading

0 comments on commit 1445acf

Please sign in to comment.