Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CW-827 CI/CD update #1948

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

CW-827 CI/CD update #1948

wants to merge 2 commits into from

Conversation

MrCyjaneK
Copy link
Collaborator

@MrCyjaneK MrCyjaneK commented Jan 7, 2025

Description

  • reduce clean build times (3h -> 1h (without monero_c cleanup which cuts the time in half))
  • reduce cached build times (35 minutes -> ~7 minutes)
  • add windows ci (WIP)
  • run on push and not on pull request
  • Windows Dockerfile is in "WIP" stage, what's inside can be used to build cake, but doesn't work for reasons that escape me on CI/CD. Separate PR will bring it into working state.
  • Added commands to commit messages, in addition to [skip ci] there is also [skip slack] - to run the build but don't send the message to slack and [run tests] which runs test (but doesn't work currently due to tests failing - also separate PR will fix that as it's not CI issue)
  • As for the cache:
    • The cache adds up to ~45GiB for a clean build. (+11GiB of preinstalled dependencies in Dockerfile). Things that get cached are as follows (in random order):
    • monero_c dependencies (i.e. boost and friends), so even monero_c updates should be quick to appear in changes and only absolutely necessary changes will get built
    • monero_c sources - so all downloads
    • go's pkg - to speedup gomobile / mwebd build
    • gradle - to speedup final android builds
    • android sdk - no installs at runtime
    • dart's pub cache
    • flutter's sdk
    • .ccache - to speedup everything that wasn't sped up in other ways (especially monero_c builds)
    • .go-cache - same as ccache but for go
    • .gradle - all .jar downloads, and transforms, this gets us a bunch of network and CPU time
    • android studio sdk dependencies
    • key.jks - only 4KiB of cache but we can update individual builds
    • preinstalled java
    • preinstalled flutter
    • preinstalled emulator and AVD image for tests

There are 4 nodes deployed currently, ci-cake-ubuntu{1-4} each with 16vCPU and 16GiB of RAM, and 256GiB of NVMe storage (free to use all of that, and if we ever need more I can bump it), and nested KVM enabled. Host is not prepared to run anything, instead we use container approach, so in addition to runs-on: linux-amd64 we also add container:.

Each node has individual cache, so when updating major dependencies we may see some weird behavior with how cache behaves (i.e. up to 4 longer builds).

Pull Request - Checklist

  • Initial Manual Tests Passed
  • Double check modified code and verify it with the feature/task requirements
  • Format code
  • Look for code duplication
  • Clear naming for variables and methods

@MrCyjaneK MrCyjaneK force-pushed the ci-test branch 4 times, most recently from ad66ac2 to 73daa2e Compare January 9, 2025 08:24
@MrCyjaneK MrCyjaneK marked this pull request as ready for review January 9, 2025 09:00
- use existing build outputs in build_monero_all.sh
- update $HOME, fix gh actions
- add secrets earlier in the runtime (potentially speed up 'Build generated code' step)
- add windows dockerfile
- add linux/android dockerfile
- update android/linux ci script
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant