Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port to rattler-build #1796

Merged
merged 63 commits into from
Feb 13, 2025
Merged

Port to rattler-build #1796

merged 63 commits into from
Feb 13, 2025

Conversation

gforsyth
Copy link
Contributor

@gforsyth gforsyth commented Jan 27, 2025

Summary:

recipe.yaml

build_*.sh

xref: rapidsai/build-planning#47

Copy link

copy-pr-bot bot commented Jan 27, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions bot added the ci label Jan 29, 2025
@gforsyth gforsyth marked this pull request as ready for review January 29, 2025 19:48
@gforsyth gforsyth requested review from a team as code owners January 29, 2025 19:48
@gforsyth
Copy link
Contributor Author

@jakirkham @vyasr anything else you'd like to see here?

Copy link
Member

@jakirkham jakirkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Gil! 🙏

Am wondering if we can simplify the templating with a context variable

conda/recipes/rmm/recipe.yaml Show resolved Hide resolved
conda/recipes/rmm/recipe.yaml Outdated Show resolved Hide resolved
@github-actions github-actions bot added the Python Related to RMM Python API label Feb 11, 2025
Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last small set of questions, then this LGTM!

ci/build_cpp.sh Outdated Show resolved Hide resolved
ci/build_cpp.sh Show resolved Hide resolved
conda/recipes/librmm/recipe.yaml Outdated Show resolved Hide resolved
@vyasr
Copy link
Contributor

vyasr commented Feb 12, 2025

With this migration and rapidsai/gha-tools#143, have we effectively obsoleted rapids-configure-conda-channels? It sounds like yes based on the --override-channels behavior that you described, but please correct me if I'm wrong. If I'm not, should we get deprecate that tool and remove it once the rattler-build migration is complete?

@vyasr
Copy link
Contributor

vyasr commented Feb 12, 2025

Since we do not, in general, use variant builds in RAPIDS (with the notable exception of ucxx), once we drop CUDA 11 support (i.e. once the compiler version is no longer dependent on the CUDA version) should we also consider inlining the CBC values? If so, can we open an issue to track that work?

@vyasr
Copy link
Contributor

vyasr commented Feb 12, 2025

I've resolved all threads that I think have been addressed (which is almost all of them). One more pass through old CI reviews to verify would be good, then I think we've got pretty much everything sorted out here.

@gforsyth
Copy link
Contributor Author

With this migration and rapidsai/gha-tools#143, have we effectively obsoleted rapids-configure-conda-channels? It sounds like yes based on the --override-channels behavior that you described, but please correct me if I'm wrong. If I'm not, should we get deprecate that tool and remove it once the rattler-build migration is complete?

I believe that's correct, so we can deprecate it and mark it for future removal.

@vyasr
Copy link
Contributor

vyasr commented Feb 13, 2025

I opened rapidsai/gha-tools#145.

Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the hard work here Gil! I think we're in good shape and ready to move on to other repos now.

ci/build_python.sh Show resolved Hide resolved
@gforsyth
Copy link
Contributor Author

Issue here to track inlining CBC values once we drop CUDA 11: rapidsai/build-planning#153

@gforsyth
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit cc969b6 into rapidsai:branch-25.04 Feb 13, 2025
62 checks passed
@jakirkham
Copy link
Member

Thanks Gil for all the hard work getting our first recipe onto rattler-build and everyone who helped review! 🙏

Looking forward to converting our next package 🙂

VenkateshJaya pushed a commit to VenkateshJaya/rmm that referenced this pull request Feb 19, 2025
Summary:

- We use the [multi-output cache](https://rattler.build/latest/multiple_output_cache/) to avoid double-compiling. The `build` environment compiles things, the individual outputs call `cmake --install`
- We make use of the built-in `git` functions for grabbing the short-SHA (https://rattler.build/latest/experimental_features/#git-functions)
- We use `load_from_file` to pull in metadata from the corresponding `pyproject.toml` (https://rattler.build/latest/experimental_features/#load_from_filefile_path)
- Relatively "simple" `*_build.sh` scripts are inlined into `recipe.yaml` instead of existing as separate files

- We use `--no-build-id` to allow `sccache` to look in a predictable place, see: https://rattler.build/latest/tips_and_tricks/#using-sccache-or-ccache-with-rattler-build
- Depending on whether `rapids-is-release-build`, we include either `rapidsai` (release) or `rapidsai-nightly` (non-release) in the channel listing
- Channels must be specified at the command-line
  - This uses https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-rattler-channel-string to generate an array of channels
- We remove the `build_cache` directory after building so it doesn't get packaged up with the other artifacts and uploaded to S3

xref: rapidsai/build-planning#47

Authors:
  - Gil Forsyth (https://github.com/gforsyth)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: rapidsai#1796
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci conda improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Related to RMM Python API
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

5 participants