Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dependencies.yaml for rapids-dependency-file-generator [skip gpuci] #5003

Merged
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
0ea34be
use AJ's cudf yaml as a skeleton/starting point
beckernick Nov 16, 2022
7aa7ce7
first pass at separated dependencies
beckernick Nov 16, 2022
bd72dfd
remove extra buildtime gcc requirements from runtime
beckernick Nov 16, 2022
72e1ba8
python and joblib
beckernick Nov 17, 2022
a6a6163
Merge branch 'branch-22.12' into feature/gh-actions-dependencies
beckernick Nov 17, 2022
4289e52
full set of CTKs, dependency updates, and comments for open questions
beckernick Nov 17, 2022
5d31390
formatting
beckernick Nov 17, 2022
3aa92ea
add generated environment files
beckernick Nov 17, 2022
53833ba
rm unused YAML anchors
ajschmidt8 Nov 18, 2022
2d134c4
rm existing `conda/environment` files
ajschmidt8 Nov 18, 2022
cc2251e
add GH Action workflow
ajschmidt8 Nov 18, 2022
93ee470
update README references
ajschmidt8 Nov 18, 2022
1a49cc8
update `update-version.sh`
ajschmidt8 Nov 18, 2022
89063ed
mv conda exclusive pkgs to conda only lists
ajschmidt8 Nov 18, 2022
8223714
mv python range to `py_version` list as default
ajschmidt8 Nov 18, 2022
2f27957
Merge remote-tracking branch 'upstream/branch-22.12' into feature/gh-…
ajschmidt8 Nov 18, 2022
6395858
update `cmake` in `dependencies.yaml`
ajschmidt8 Nov 18, 2022
33bd3a6
update generated dependency files
ajschmidt8 Nov 18, 2022
28c504f
rm comments
ajschmidt8 Nov 22, 2022
d76c5ef
commit latest `dependencies.yaml` changes
ajschmidt8 Nov 22, 2022
1e31dfc
Merge branch-23.02 into feature/gh-actions-dependencies
ajschmidt8 Dec 1, 2022
088af05
UPD Update versions to 23.02
dantegd Dec 6, 2022
7859549
Apply suggestions from code review
ajschmidt8 Dec 7, 2022
e5a9c39
Merge remote-tracking branch 'upstream/branch-23.02' into feature/gh-…
ajschmidt8 Dec 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/pr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: pr

on:
pull_request:

jobs:
checks:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@main
with:
enable_check_size: false
enable_check_style: false
4 changes: 2 additions & 2 deletions BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ To install cuML from source, ensure the following dependencies are met:
9. NCCL (>=2.4)
10. UCX [optional] (>= 1.7) - enables point-to-point messaging in the cuML standard communicator. This is necessary for many multi-node multi-GPU cuML algorithms to function.

It is recommended to use conda for environment/package management. If doing so, a convenience environment .yml file is located in `conda/environments/cuml_dec_cudax.y.yml` (replace x.y for your CUDA version). This file contains most of the dependencies mentioned above (notable exceptions are `gcc` and `zlib`). To use it, for example to create an environment named `cuml_dev` for CUDA 10.2 and Python 3.7, you can use the follow command:
It is recommended to use conda for environment/package management. If doing so, development environment .yaml files are located in `conda/environments/all_*.yaml`. These files contains most of the dependencies mentioned above (notable exceptions are `gcc` and `zlib`). To use them, for example to create an environment named `cuml_dev` for CUDA 10.2 and Python 3.7, you can use the follow command:
ajschmidt8 marked this conversation as resolved.
Show resolved Hide resolved

```bash
conda create -n cuml_dev python=3.8
conda activate cuml_dev
conda env update --file=conda/environments/cuml_dev_cuda11.2.yml
conda env update --file=conda/environments/all_cuda-115_arch-x86_64.yaml
```

These conda environments are based on the general RAPIDS meta packages that install common dependencies for RAPIDS projects. To install different versions of packages contained in those meta packages after creating the environment, it is recommended to remove those meta packages (without removing the actual packages contained in the environment) with the following command (having the environment active):
Expand Down
4 changes: 3 additions & 1 deletion ci/release/update-version.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ sed_runner 's/'"branch-.*\/RAPIDS.cmake"'/'"branch-${NEXT_SHORT_TAG}\/RAPIDS.cma
sed_runner 's/version = .*/version = '"'${NEXT_SHORT_TAG}'"'/g' docs/source/conf.py
sed_runner 's/release = .*/release = '"'${NEXT_FULL_TAG}'"'/g' docs/source/conf.py

for FILE in conda/environments/*.yml; do
for FILE in dependencies.yaml conda/environments/*.yaml; do
sed_runner "s/cudf=${CURRENT_SHORT_TAG}/cudf=${NEXT_SHORT_TAG}/g" ${FILE};
sed_runner "s/rmm=${CURRENT_SHORT_TAG}/rmm=${NEXT_SHORT_TAG}/g" ${FILE};
sed_runner "s/dask-cuda=${CURRENT_SHORT_TAG}/dask-cuda=${NEXT_SHORT_TAG}/g" ${FILE};
Expand All @@ -62,3 +62,5 @@ done
sed_runner "s/export UCX_PY_VERSION=.*/export UCX_PY_VERSION='${NEXT_UCX_PY_VERSION}'/g" ci/checks/style.sh
sed_runner "s/export UCX_PY_VERSION=.*/export UCX_PY_VERSION='${NEXT_UCX_PY_VERSION}'/g" ci/cpu/build.sh
sed_runner "s/export UCX_PY_VERSION=.*/export UCX_PY_VERSION='${NEXT_UCX_PY_VERSION}'/g" ci/gpu/build.sh

sed_runner "s|/branch-.*/|/branch-${NEXT_SHORT_TAG}/|g" python/README.md
53 changes: 53 additions & 0 deletions conda/environments/all_cuda-115_arch-x86_64.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# This file is generated by `rapids-dependency-file-generator`.
# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
channels:
- rapidsai
- rapidsai-nightly
- dask/label/dev
- conda-forge
- nvidia
dependencies:
- cmake>=3.23.1,!=3.25.0
- cuda-python>=11.7.1,<12.0
- cudatoolkit=11.5
- cudf=22.12.*
- cupy>=7.8.0,<12.0.0a0
- cython>=0.29,<0.30
- dask-cuda=22.12.*
- dask-cudf=22.12.*
- dask-ml
- dask>=2022.9.2
- distributed>=2022.9.2
- doxygen>=1.8.20
- faiss-proc=*=cuda
- gcc_linux-64=9.*
- joblib>=0.11
- libcumlprims=22.12.*
- libfaiss>=1.7.0
- libraft-distance=22.12.*
- libraft-headers=22.12.*
- libraft-nn=22.12.*
- nccl>=2.9.9
- nltk
- nvcc_linux-64=11.5
- pip
- pylibraft=22.12.*
- python>=3.8,<3.10
- raft-dask=22.12.*
- rmm=22.12.*
- scikit-build>=0.13.1
- scikit-learn=0.24
- seaborn
- sphinx-markdown-tables
- statsmodels
- sysroot_linux-64==2.17
- treelite=3.0.1
- ucx-proc=*=gpu
- ucx-py=0.29.*
- ucx>=1.13.0
- umap-learn
- pip:
- git+https://github.com/dask/dask-glm@main
- git+https://github.com/scikit-learn-contrib/hdbscan.git@master
- sparse
name: all_cuda-115_arch-x86_64
22 changes: 22 additions & 0 deletions conda/environments/cpp_all_cuda-115_arch-x86_64.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# This file is generated by `rapids-dependency-file-generator`.
# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
channels:
- rapidsai
- rapidsai-nightly
- dask/label/dev
- conda-forge
- nvidia
dependencies:
- cmake>=3.23.1,!=3.25.0
- cudatoolkit=11.5
- faiss-proc=*=cuda
- gcc_linux-64=9.*
- libcumlprims=22.12.*
- libfaiss>=1.7.0
- libraft-distance=22.12.*
- libraft-headers=22.12.*
- libraft-nn=22.12.*
- nvcc_linux-64=11.5
- rmm=22.12.*
- sysroot_linux-64==2.17
name: cpp_all_cuda-115_arch-x86_64
54 changes: 0 additions & 54 deletions conda/environments/cuml_dev_cuda11.0.yml

This file was deleted.

53 changes: 0 additions & 53 deletions conda/environments/cuml_dev_cuda11.2.yml

This file was deleted.

53 changes: 0 additions & 53 deletions conda/environments/cuml_dev_cuda11.4.yml

This file was deleted.

53 changes: 0 additions & 53 deletions conda/environments/cuml_dev_cuda11.5.yml

This file was deleted.

Loading