Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor how linux_x64_clang_debug uses Docker and scripts. #18255

Merged
merged 3 commits into from
Aug 19, 2024

Conversation

ScottTodd
Copy link
Member

@ScottTodd ScottTodd commented Aug 16, 2024

Progress on #15332 and #18238 .

Similar to #18252, this drops a dependency on the build_tools/docker/docker_run.sh script. Unlike that PR, this goes a step further and also stops using build_tools/cmake/build_all.sh.

Functional changes:

  • No more building iree-test-deps
  • No more ccache
    • The debug build cache is too large for a local / GitHub Actions cache
    • I want to limit our reliance on the remote cache at http://storage.googleapis.com/iree-sccache/ccache (which uses GCP for storage and needs GCP auth)
    • Experiments show that this build is not significantly faster when using a cache, or at least dropping iree-test-deps provides equivalent time savings

Logs before: https://github.com/iree-org/iree/actions/runs/10417779910/job/28864909582 (96% cache hits, 9 minute build but 19 minutes total, due to iree-test-deps)
Logs after: https://github.com/iree-org/iree/actions/runs/10423409599/job/28870060781?pr=18255 (no cache, 11 minute build)

ci-exactly: linux_x64_clang_debug

@ScottTodd ScottTodd added infrastructure Relating to build systems, CI, or testing cleanup 🧹 labels Aug 16, 2024
@ScottTodd ScottTodd marked this pull request as ready for review August 16, 2024 17:09
@ScottTodd ScottTodd requested review from benvanik and marbre August 16, 2024 17:09
Comment on lines 55 to 56
# Note: not using ccache here. Debug builds need a lot of cache space
# and caching only provides marginal build time improvements.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a 2GB cache getting 11 / 5262 cache hits:
https://github.com/iree-org/iree/actions/runs/10411642302/job/28835942071#step:15:6

Soooo not possible using these cache settings and a local cache (backed up GitHub Actions Cache for save/restore). Might be able to use different settings like sccache or more compression, but it seems like the cache only saves 2 minutes on an 11 minute build for -DCMAKE_BUILD_TYPE=Debug anyways 🤷‍♂️

Copy link
Member

@marbre marbre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reduced build time, yeah!

From an outsiders perspective it might not be obvious to understand why only building the project without running is useful. As the samples get build the compiler is used to compile those, but it may not be immediately clear. Thus, we could think about documenting what kind of workflows we're running why at some point.

.github/workflows/ci_linux_x64_clang_debug.yml Outdated Show resolved Hide resolved
@ScottTodd
Copy link
Member Author

Reduced build time, yeah!

From an outsiders perspective it might not be obvious to understand why only building the project without running is useful. As the samples get build the compiler is used to compile those, but it may not be immediately clear. Thus, we could think about documenting what kind of workflows we're running why at some point.

https://iree.dev/developers/general/github-actions/#other-build-configurations

Other build configurations

These workflows build the full project from source using optional settings then run basic tests.

  • Workflows in this category can use sanitizers, debug builds, alternate compilers, and other features that maintainers want automated coverage for.

For this 'debug' build, we sometimes see build issues only appear in debug (NDEBUG set/unset). Runtime failures unique to debug are also possible, but I think we could add a few smoketests for that instead of relying on building the iree-test-deps target.

Co-authored-by: Marius Brehler <[email protected]>
@ScottTodd ScottTodd enabled auto-merge (squash) August 19, 2024 15:27
@ScottTodd ScottTodd merged commit b637a30 into iree-org:main Aug 19, 2024
24 checks passed
@ScottTodd ScottTodd deleted the infra-docker-run-workflows-2 branch August 19, 2024 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup 🧹 infrastructure Relating to build systems, CI, or testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants