Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

prefer system installs of UCX libraries when using RAPIDS wheels #421

Merged
merged 3 commits into from
Nov 27, 2024
Merged

prefer system installs of UCX libraries when using RAPIDS wheels #421

merged 3 commits into from
Nov 27, 2024

Conversation

jameslamb
Copy link
Member

Contributes to rapidsai/build-planning#118

Should only be merged if / after we merge rapidsai/ucx-wheels#13. That PR switches libucx wheels' loading behavior, such that libucx.load_library() (which ucx-py and libucxx call at import time) prefers the libuc{m,p,s}.so provided by the libucx wheels.

rapidsai/ucx-wheels#13 introduces an environment variable to control that... this proposes setting that, to continue preferring system-installed UCX libraries in devcontainers.

@jameslamb jameslamb requested review from trxcllnt and vyasr November 15, 2024 19:25
@jameslamb jameslamb requested a review from a team as a code owner November 15, 2024 19:25
@jameslamb jameslamb mentioned this pull request Nov 15, 2024
Closed
@vyasr
Copy link
Contributor

vyasr commented Nov 18, 2024

@trxcllnt does this look fine to you?

trxcllnt
trxcllnt requested changes Nov 18, 2024
View reviewed changes
Copy link
Collaborator

@trxcllnt trxcllnt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put this envvar in the pip section of the respective devcontainers' Dockerfiles?

features/src/rapids-build-utils/devcontainer-feature.json Outdated Show resolved Hide resolved
@vyasr vyasr requested a review from trxcllnt November 22, 2024 19:07
@jameslamb
Copy link
Member Author

Can we put this envvar in the pip section of the respective devcontainers' Dockerfiles?

I think @vyasr 's change here meets this request, @trxcllnt could you please look again?

@trxcllnt
Copy link
Collaborator

LGTM. This covers the combined devcontainer in this repo, but we should also add this to any repos that use UCX in their pip devcontainers.

@trxcllnt trxcllnt merged commit e37f7b0 into rapidsai:branch-24.12 Nov 27, 2024
24 checks passed
@jameslamb jameslamb deleted the ucx-libraries branch November 27, 2024 15:39
@jameslamb
Copy link
Member Author

Alright thanks for pointing that out! I can do that. Added a note in rapidsai/build-planning#118

This was referenced Nov 27, 2024
Merged
Merged
Merged
Merged
Merged
Merged
@jameslamb
Copy link
Member Author

we should also add this to any repos that use UCX in their pip devcontainers.

put up PRs for all of those:

rapids-bot bot pushed a commit to rapidsai/nx-cugraph that referenced this pull request Dec 2, 2024
@jameslamb
prefer system install of UCX in devcontainers, update outdated RAPIDS…
1cd7cc6 
… references (#53)

Contributes to rapidsai/build-planning#118

Proposes the following changes for pip devcontainers:

* use UCX 1.17 (ref: rapidsai/cugraph-gnn#79 (comment))
* prefer system installation of ucx to the one provided by the `libucx-cu{11,12}` wheels (ref: rapidsai/devcontainers#421 (comment))

And some other related changes noticed while doing that:

* update lingering `24.*` references to `25.02`
* fix `update-version.sh` so those will be correctly updated in future releases

Similar to rapidsai/cugraph#4792

## Notes for Reviewers

### How I tested this

Relying on CI for most things. But for `update-version.sh`, tested like this:

```shell
./ci/release/update-version.sh '25.02.00'
git grep -E '24\.'
```

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #53
rapids-bot bot pushed a commit to rapidsai/cuvs that referenced this pull request Dec 2, 2024
@jameslamb
prefer system install of UCX in devcontainers (#501)
1215882 
Contributes to rapidsai/build-planning#118

Proposes the following changes for pip devcontainers:

* prefer system installation of ucx to the one provided by the `libucx-cu{11,12}` wheels (ref: rapidsai/devcontainers#421 (comment))

And some other related changes noticed while doing that:

* update lingering `24.*` references to `25.02`

## Notes for Reviewers

### How I tested this

Relying on CI for most things. Double-checked that `update-version.sh` would have caught the one lingering `24.12` reference like this:

```shell
./ci/release/update-version.sh '25.02.00'
git grep -E '24\.'
```

Similar to rapidsai/cuml#6149

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #501
rapids-bot bot pushed a commit to rapidsai/cugraph that referenced this pull request Dec 2, 2024
@jameslamb
pip devcontainers: use UCX 1.17, prefer system installation (#4792)
94c6ee4 
Contributes to rapidsai/build-planning#118

Proposes the following changes for pip devcontainers:

* use UCX 1.17 (ref: rapidsai/cugraph-gnn#79 (comment))
* prefer system installation of ucx to the one provided by the `libucx-cu{11,12}` wheels (ref: rapidsai/devcontainers#421 (comment))

And one other related change:

* skip most CI when a PR only changes files in the `.devcontainer/` directory (this was incorrectly spelled `.devcontainers/` before)

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)

URL: #4792
rapids-bot bot pushed a commit to rapidsai/raft that referenced this pull request Dec 4, 2024
@jameslamb
prefer system install of UCX in devcontainers, update outdated RAPIDS…
fc7818f 
… references (#2514)

Contributes to rapidsai/build-planning#118

Proposes the following changes for pip devcontainers:

* prefer system installation of ucx to the one provided by the `libucx-cu{11,12}` wheels (ref: rapidsai/devcontainers#421 (comment))

And some other related changes noticed while doing that:

* update lingering `24.*` references to `25.02`

## Notes for Reviewers

### How I tested this

Relying on CI for most things. Double-checked that `update-version.sh` would have caught the one lingering `24.12` reference like this:

```shell
./ci/release/update-version.sh '25.02.00'
git grep -E '24\.'
```

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #2514
rapids-bot bot pushed a commit to rapidsai/cuml that referenced this pull request Dec 11, 2024
@jameslamb
prefer system install of UCX in pip devcontainers, update outdated RA…
052cdde 
…PIDS references (#6149)

Contributes to rapidsai/build-planning#118

Proposes the following changes for pip devcontainers:

* prefer system installation of ucx to the one provided by the `libucx-cu{11,12}` wheels (ref: rapidsai/devcontainers#421 (comment))

And some other related changes noticed while doing that:

* update lingering `24.*` references to `25.02`

## Notes for Reviewers

### How I tested this

Relying on CI for most things. Double-checked that `update-version.sh` would have caught the one lingering `24.12` reference like this:

```shell
./ci/release/update-version.sh '25.02.00'
git grep -E '24\.'
```

Authors:
  - James Lamb (https://github.com/jameslamb)
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Tim Head (https://github.com/betatim)
  - Bradley Dice (https://github.com/bdice)

URL: #6149
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@trxcllnt trxcllnt trxcllnt approved these changes

@vyasr vyasr Awaiting requested review from vyasr

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jameslamb @vyasr @trxcllnt