Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update RAPIDS dependencies to 24.4, refactor dependencies.yaml #5726

Merged
merged 8 commits into from
Jan 24, 2024
Merged

update RAPIDS dependencies to 24.4, refactor dependencies.yaml #5726

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3110929
update RAPIDS dependencies to 24.4
jameslamb Jan 22, 2024
d7be0cb
fix update-version.sh
jameslamb Jan 22, 2024
e61bbff
refactor dependencies
jameslamb Jan 22, 2024
a74289a
Update ci/release/update-version.sh
jameslamb Jan 22, 2024
8cf9ec9
more updates
jameslamb Jan 22, 2024
10d0c63
empty commit to re-trigger CI
jameslamb Jan 23, 2024
926f617
debug git and ssh configs
trxcllnt Jan 24, 2024
f0a9ed4
revert previous commit
trxcllnt Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/release/update-version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ DEPENDENCIES=(
)
for FILE in dependencies.yaml conda/environments/*.yaml; do
for DEP in "${DEPENDENCIES[@]}"; do
sed_runner "/- ${DEP}==/ s/==.*/==${NEXT_SHORT_TAG_PEP440}\.*/g" ${FILE};
sed_runner "/-.* ${DEP}\(-cu[[:digit:]]\{2\}\)\{0,1\}==/ s/==.*/==${NEXT_SHORT_TAG_PEP440}.*/g" "${FILE}"
done
done

Expand Down
10 changes: 5 additions & 5 deletions conda/environments/all_cuda-118_arch-x86_64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ dependencies:
- cuda-python>=11.7.1,<12.0a0
- cuda-version=11.8
- cudatoolkit
- cudf==24.2.*
- cudf==24.4.*
- cupy>=12.0.0
- cxx-compiler
- cython>=3.0.0
- dask-cuda==24.4.*
- dask-cudf==24.2.*
- dask-cudf==24.4.*
- dask-ml
- doxygen=1.9.1
- gcc_linux-64=11.*
Expand Down Expand Up @@ -51,18 +51,18 @@ dependencies:
- nvcc_linux-64=11.8
- pip
- pydata-sphinx-theme!=0.14.2
- pylibraft==24.2.*
- pylibraft==24.4.*
- pynndescent==0.5.8
- pytest
- pytest-benchmark
- pytest-cases
- pytest-cov
- pytest-xdist
- python>=3.9,<3.11
- raft-dask==24.2.*
- raft-dask==24.4.*
- rapids-dask-dependency==24.4.*
- recommonmark
- rmm==24.2.*
- rmm==24.4.*
- scikit-build-core>=0.7.0
- scikit-learn==1.2
- scipy>=1.8.0
Expand Down
10 changes: 5 additions & 5 deletions conda/environments/all_cuda-120_arch-x86_64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ dependencies:
- cuda-profiler-api
- cuda-python>=12.0,<13.0a0
- cuda-version=12.0
- cudf==24.2.*
- cudf==24.4.*
- cupy>=12.0.0
- cxx-compiler
- cython>=3.0.0
- dask-cuda==24.4.*
- dask-cudf==24.2.*
- dask-cudf==24.4.*
- dask-ml
- doxygen=1.9.1
- gcc_linux-64=11.*
Expand Down Expand Up @@ -47,18 +47,18 @@ dependencies:
- numpydoc
- pip
- pydata-sphinx-theme!=0.14.2
- pylibraft==24.2.*
- pylibraft==24.4.*
- pynndescent==0.5.8
- pytest
- pytest-benchmark
- pytest-cases
- pytest-cov
- pytest-xdist
- python>=3.9,<3.11
- raft-dask==24.2.*
- raft-dask==24.4.*
- rapids-dask-dependency==24.4.*
- recommonmark
- rmm==24.2.*
- rmm==24.4.*
- scikit-build-core>=0.7.0
- scikit-learn==1.2
- scipy>=1.8.0
Expand Down
60 changes: 26 additions & 34 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,8 +145,7 @@ dependencies:
packages:
- nvcc_linux-aarch64=11.8
- matrix:
arch: x86_64
cuda: "12.0"
cuda: "12.*"
packages:
- cuda-nvcc
py_build:
Expand All @@ -157,8 +156,8 @@ dependencies:
- &treelite treelite==3.9.1
- output_types: conda
packages:
- &pylibraft_conda pylibraft==24.2.*
- &rmm_conda rmm==24.2.*
- &pylibraft_conda pylibraft==24.4.*
- &rmm_conda rmm==24.4.*
Comment on lines +159 to +160
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know why were these missed by update-version.sh previously? They're not -cu11 or -cu12 names. Was it due to a forward-merge after branch-24.04 was created?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The &rmm_conda part prevented being updated. That's why the line in update-version.sh added the .* as in -.* ${DEP}...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me!

The presence of - in these commands also explains why some projects have separate lines for updating pyproject.toml files, like this:

https://github.com/rapidsai/cugraph/blob/f20219ed62aaade9ff21e5867ea828ee20baef9b/ci/release/update-version.sh#L97-L107

- scikit-build-core>=0.7.0
- output_types: requirements
packages:
Expand All @@ -174,25 +173,22 @@ dependencies:
- output_types: [conda, requirements, pyproject]
matrices:
- matrix:
cuda: "12.0"
cuda: "12.*"
packages:
- &cuda_python12 cuda-python>=12.0,<13.0a0
- cuda-python>=12.0,<13.0a0
- matrix: # All CUDA 11 versions
packages:
- &cuda_python11 cuda-python>=11.7.1,<12.0a0
- cuda-python>=11.7.1,<12.0a0
- output_types: [requirements, pyproject]
matrices:
- matrix: {cuda: "12.0"}
- matrix: {cuda: "12.*"}
packages:
- pylibraft-cu12==24.4.*
- rmm-cu12==24.4.*
- matrix: {cuda: "11.*"}
packages:
- pylibraft-cu12==24.2.*
- rmm-cu12==24.2.*
- matrix: {cuda: "11.8"}
packages: &py_build_packages_cu11
- &pylibraft_cu11 pylibraft-cu11==24.2.*
- &rmm_cu11 rmm-cu11==24.2.*
- {matrix: {cuda: "11.5"}, packages: *py_build_packages_cu11}
- {matrix: {cuda: "11.4"}, packages: *py_build_packages_cu11}
- {matrix: {cuda: "11.2"}, packages: *py_build_packages_cu11}
- &pylibraft_cu11 pylibraft-cu11==24.4.*
- &rmm_cu11 rmm-cu11==24.4.*
- {matrix: null, packages: [*pylibraft_conda, *rmm_conda] }

py_run:
Expand All @@ -210,10 +206,10 @@ dependencies:
- *treelite
- output_types: conda
packages:
- &cudf_conda cudf==24.2.*
- &cudf_conda cudf==24.4.*
- &cupy_conda cupy>=12.0.0
- &dask_cudf_conda dask-cudf==24.2.*
- &raft_dask_conda raft-dask==24.2.*
- &dask_cudf_conda dask-cudf==24.4.*
- &raft_dask_conda raft-dask==24.4.*
- output_types: requirements
packages:
# pip recognizes the index as a global option for the requirements.txt file
Expand All @@ -226,26 +222,22 @@ dependencies:
specific:
- output_types: [requirements, pyproject]
matrices:
- matrix: {cuda: "12.0"}
- matrix: {cuda: "12.*"}
packages:
- cudf-cu12==24.2.*
- cudf-cu12==24.4.*
- cupy-cuda12x>=12.0.0
- dask-cudf-cu12==24.2.*
- pylibraft-cu12==24.2.*
- raft-dask-cu12==24.2.*
- rmm-cu12==24.2.*
# All CUDA 11 versions
- matrix: {cuda: "11.8"}
- dask-cudf-cu12==24.4.*
- pylibraft-cu12==24.4.*
- raft-dask-cu12==24.4.*
- rmm-cu12==24.4.*
- matrix: {cuda: "11.*"}
packages: &py_run_packages_cu11
- cudf-cu11==24.2.*
- cudf-cu11==24.4.*
- &cupy_pyproject_cu11 cupy-cuda11x>=12.0.0
- dask-cudf-cu11==24.2.*
- dask-cudf-cu11==24.4.*
- *pylibraft_cu11
- raft-dask-cu11==24.2.*
- raft-dask-cu11==24.4.*
- *rmm_cu11
- {matrix: {cuda: "11.5"}, packages: *py_run_packages_cu11}
- {matrix: {cuda: "11.4"}, packages: *py_run_packages_cu11}
- {matrix: {cuda: "11.2"}, packages: *py_run_packages_cu11}
- matrix: null
packages:
- *cudf_conda
Expand Down
14 changes: 7 additions & 7 deletions python/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ requires = [
"cuda-python>=11.7.1,<12.0a0",
"cython>=3.0.0",
"ninja",
"pylibraft==24.2.*",
"rmm==24.2.*",
"pylibraft==24.4.*",
"rmm==24.4.*",
"scikit-build-core[pyproject]>=0.7.0",
"treelite==3.9.1",
"treelite_runtime==3.9.1",
Expand Down Expand Up @@ -55,16 +55,16 @@ authors = [
license = { text = "Apache 2.0" }
requires-python = ">=3.9"
dependencies = [
"cudf==24.2.*",
"cudf==24.4.*",
"cupy-cuda11x>=12.0.0",
"dask-cuda==24.4.*",
"dask-cudf==24.2.*",
"dask-cudf==24.4.*",
"joblib>=0.11",
"numba>=0.57",
"pylibraft==24.2.*",
"raft-dask==24.2.*",
"pylibraft==24.4.*",
"raft-dask==24.4.*",
"rapids-dask-dependency==24.4.*",
"rmm==24.2.*",
"rmm==24.4.*",
"scipy>=1.8.0",
"treelite==3.9.1",
"treelite_runtime==3.9.1",
Expand Down