Skip to content

Commit

Permalink
ci: Cache dependencies to speed up test runs (sourcenetwork#2732)
Browse files Browse the repository at this point in the history
## Relevant issue(s)
Resolves sourcenetwork#2735 

## Description
In this PR we cache go and rust build/dep states, and restore them if an
action run has same dependency graph. Go cache is around 877 MB and
cargo/rust cache is about 189MB (99MB for macos) so we can roughly have
room for 5 caches (x1 for linux and x1 for macos) before a cache is
dropped / lost.

Notes:
- Github gives us 10GB of cache storage by default.
- Rather than going with the standard practice of using lockfiles to
generate the caching key we will use the mod and toml files, in order to
maximize cache hits. Incase troublesome we can go back to lockfiles.
- This change does not help the change detector. IMO change detector
should move to a separate job, or a new workflow file. (opened
sourcenetwork#2736)


### Demo
- Without cache run (current state):
https://github.com/sourcenetwork/defradb/actions/runs/9554784255?pr=2732
<img
src="https://github.com/sourcenetwork/defradb/assets/30120428/c0b7720a-bd3f-4a01-853f-43d7b135ab76"
width="400" />


- With cache run (after this PR):
https://github.com/sourcenetwork/defradb/actions/runs/9555174699
<img
src="https://github.com/sourcenetwork/defradb/assets/30120428/db9d0bab-7fcb-4255-99f1-bd1d83220795"
width="400" />




## How has this been tested?
- Manual
- `act` tool
  • Loading branch information
shahzadlone authored Jun 18, 2024
1 parent 38f9b9f commit 3c0a14a
Showing 1 changed file with 39 additions and 0 deletions.
39 changes: 39 additions & 0 deletions .github/workflows/test-and-upload-coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,45 @@ jobs:
go-version: "1.21"
check-latest: true

- name: Set cache paths
id: cache-paths
shell: bash
run: |
echo "GO_CACHE=$(go env GOCACHE)" >> "${GITHUB_OUTPUT}"
echo "GO_MODCACHE=$(go env GOMODCACHE)" >> "${GITHUB_OUTPUT}"
echo "CARGO_CACHE=~/.cargo" >> "${GITHUB_OUTPUT}"
- name: Go cache/restore
uses: actions/cache@v4
with:
key: ${{ runner.os }}-go-${{ hashFiles('**/go.mod') }}
path: |
${{ steps.cache-paths.outputs.GO_CACHE }}
${{ steps.cache-paths.outputs.GO_MODCACHE }}
- name: Cargo cache/restore
# A very cool post: https://blog.arriven.wtf/posts/rust-ci-cache
uses: actions/cache@v4
with:
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.toml') }}
# Here are some directories we shouldn't forget about:
# ~/.cargo/.*
# ~/.cargo/bin/
# ~/.cargo/git/db/
# ~/.cargo/registry/cache/
# ~/.cargo/registry/index/
# **/target/*/*.d
# **/target/*/*.rlib
# **/target/*/.fingerprint
# **/target/*/build
# **/target/*/deps
path: |
${{ steps.cache-paths.outputs.CARGO_CACHE }}
**/target/
- name: Restore modified time
uses: chetan/git-restore-mtime-action@v2

- name: Build dependencies
run: |
make deps:modules
Expand Down

0 comments on commit 3c0a14a

Please sign in to comment.