Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build wheels without build isolation #1473

Merged
merged 31 commits into from
Oct 22, 2024

Conversation

jameslamb
Copy link
Member

@jameslamb jameslamb commented Oct 11, 2024

Description

Contributes to rapidsai/build-planning#108

Contributes to rapidsai/build-planning#111

Proposes building libcuspatial wheels with --no-build-isolation, to improve the rate of sccache cache hits and therefore reduce CI times.

Also proposes printing sccache stats to CI logs after wheel and conda builds.

Notes for Reviewers

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

How I tested this

Ran a first build with these changes to populate the cache, then ran again... saw a 100% cache hit rate and the actual wheel-building part of a wheel-build-libcuspatial job take only 37 seconds.

Successfully built libcuspatial-cu11
Compile requests                     28
Compile requests executed            28
Cache hits                           28
Cache hits (c [gcc])                  2
Cache hits (c++ [gcc])                2
Cache hits (cuda [nvcc])             24

(build link)

@jameslamb jameslamb added 2 - In Progress Currenty a work in progress 5 - DO NOT MERGE Hold off on merging; see PR for details labels Oct 11, 2024
@github-actions github-actions bot added the ci label Oct 11, 2024
@jameslamb jameslamb added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currenty a work in progress 5 - DO NOT MERGE Hold off on merging; see PR for details labels Oct 14, 2024
@jameslamb jameslamb changed the title WIP: [DO NOT MERGE] test wheel-build sccache fixes build wheels without build isolation Oct 14, 2024
@jameslamb jameslamb added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 14, 2024
@jameslamb jameslamb marked this pull request as ready for review October 14, 2024 14:56
@jameslamb jameslamb requested a review from a team as a code owner October 14, 2024 14:56
ci/build_wheel.sh Outdated Show resolved Hide resolved
@jameslamb jameslamb marked this pull request as draft October 17, 2024 18:48
@jameslamb jameslamb added 2 - In Progress Currenty a work in progress and removed 3 - Ready for Review Ready for review by team labels Oct 17, 2024
@jameslamb jameslamb changed the title build wheels without build isolation WIP: build wheels without build isolation Oct 17, 2024
@jameslamb jameslamb changed the title WIP: build wheels without build isolation build wheels without build isolation Oct 17, 2024
@jameslamb jameslamb added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currenty a work in progress 5 - DO NOT MERGE Hold off on merging; see PR for details labels Oct 17, 2024
@jameslamb jameslamb marked this pull request as ready for review October 17, 2024 19:32
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me. Just to verify, nothing is needed for cuproj since it doesn't have C++ wheels, right?

Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thanks for iterating a few times James, I think we've come up with a clean end state that we can port everywhere now.

@vyasr
Copy link
Contributor

vyasr commented Oct 22, 2024

And yes that's right Bradley, no C++ cuproj. It's built as part of libcuspatial.

RAPIDS_PACKAGE_VERSION=$(rapids-generate-version) rapids-conda-retry mambabuild \
conda/recipes/libcuspatial

sccache --show-adv-stats
Copy link
Member Author

@jameslamb jameslamb Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added sccache stats printing for conda builds as well based on rapidsai/build-planning#111 and offline conversation with @vyasr .

example build with these: https://github.com/rapidsai/cuspatial/actions/runs/11465788119/job/31906443973?pr=1473

Why not build.sh?

Here, I'm proposing that these be put in ci/build_{cpp,python}.sh, wrapped around any conda mambabuild invocations that will end up compiling anything (so, for example, I included them in build_python.sh because cuspatial and cuproj both have some Cython extensions).

I think this is preferable to build.sh, because the build_{cpp,python}.sh scripts are unambiguously for CI, and can unconditionally invoke sccache. build.sh is invoked in CI, but it's also intended for local use, and I didn't want to assume the presence of sccache for all local builds are introduce the complexity of having to check for its use.

There are also some projects, like cuvs, that have their own handling of printing sccache stats and creating build summary reports based on them in build.sh, and I didn't want to interfere with that.

example:

https://github.com/rapidsai/cuvs/blob/e7f1085b71c340b9600f5f38f7f0059a5c7aa806/build.sh#L364-L366

https://github.com/rapidsai/cuvs/blob/e7f1085b71c340b9600f5f38f7f0059a5c7aa806/build.sh#L408-L424

@jameslamb
Copy link
Member Author

Thanks for the reviews everyone! This looks good to me (double-checked that we're seeing sccache stats we expect and in all the expected places), so I'm going to merge it.

@jameslamb
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 86ef675 into rapidsai:branch-24.12 Oct 22, 2024
75 checks passed
@jameslamb jameslamb deleted the wheel-build-sccache branch October 22, 2024 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team ci improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

5 participants