Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cuml: Build CUDA 12 packages #5318

Merged
merged 44 commits into from
Jul 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
b20cc08
Switch to using CUDA 12 workflows
vyasr Mar 31, 2023
0f9aa61
Add nvcc flags needed to work with broken nvidia packages
vyasr Mar 31, 2023
b77fd91
Add a cuda11 compiler version and branch out the recipes
vyasr Mar 31, 2023
a42fb8e
Force build and test scripts to install rmm from the PR.
vyasr Mar 31, 2023
436a1f6
Comment out unnecessary builds
vyasr Mar 31, 2023
1b2ac1c
Bifurcate all cuda deps by CUDA version
vyasr Mar 31, 2023
42c530e
Pull raft and culmprims from CUDA 12 PRs
vyasr Apr 6, 2023
fb2f30e
Patch conda channels
vyasr Apr 6, 2023
9cf22b8
Update dependencies.yaml
vyasr Apr 6, 2023
6954977
Remove faiss dep for CUDA 12
vyasr Apr 6, 2023
0a44511
Update to latest raft and cumlprims hashes
vyasr Apr 6, 2023
347195b
UPD Update RMM hash
dantegd Apr 13, 2023
424ee3c
Empty commit to get CI fix
dantegd Apr 13, 2023
188c8da
Merge branch-23.06
dantegd Apr 14, 2023
d5fbb32
FIX line from bad merge
dantegd Apr 14, 2023
422b49a
DBG remove conda-forge channel packages before installing artifacts
dantegd Apr 14, 2023
c51a1d5
FIX Remove cudf from libcuml conda recipe
dantegd Apr 15, 2023
083b9e8
DBG build libcuml package in singlegpu mode
dantegd Apr 16, 2023
987eb4b
FIX remove straggling channel
dantegd Apr 16, 2023
a081d60
Solve merge conflicts
dantegd Jul 10, 2023
9641d91
UPD Update libcuml conda recipe
dantegd Jul 10, 2023
b00a69f
UPD Update cuml conda recipe
dantegd Jul 10, 2023
718fbd3
UPD dependencies.yaml updates for CUDA 12
dantegd Jul 10, 2023
851b69f
FIX remove test_cpp.sh changes
dantegd Jul 10, 2023
e950ee0
FIX Remove explicit libraft-headers from libcuml recipe
dantegd Jul 10, 2023
2a83318
FIX prefix of cuda variables in recipe
dantegd Jul 10, 2023
2bff4c1
FIX ignore_run_exports from PR review
dantegd Jul 10, 2023
4c49832
FIX add cmake build req back in libcuml recipe
dantegd Jul 10, 2023
1f1e7e3
Update conda/recipes/libcuml/meta.yaml
dantegd Jul 11, 2023
3a8532c
FIX cuml conda_config fix and gha cuda-120 actions
dantegd Jul 11, 2023
ab9c9d8
Merge branch 'branch-23.08' into feat/cuda12_nvidia_build
dantegd Jul 11, 2023
c641c53
Apply suggestions from code review
dantegd Jul 11, 2023
a8d102f
FIX update dependencies.yaml
dantegd Jul 11, 2023
85960da
FIX temporarily skip test_sparse_pca_inputs in CUDA 12
dantegd Jul 11, 2023
f40b7f9
Fix PCA test
viclafargue Jul 11, 2023
1d06c86
FIX Merge fix of PR #5498
dantegd Jul 11, 2023
325e954
FIX Remove not used import
dantegd Jul 11, 2023
b3cee15
Apply suggestions from code review
dantegd Jul 11, 2023
e107588
FIX remove dev packages from run reqs of libcuml
dantegd Jul 11, 2023
b186129
FIX forgot to run rdfg
dantegd Jul 11, 2023
94995c1
Update conda/recipes/libcuml/meta.yaml
dantegd Jul 12, 2023
a301e01
DBG remove dev ctk packages from cuml host
dantegd Jul 12, 2023
fe27991
Apply suggestions from code review
dantegd Jul 13, 2023
7ce78b4
DBG remove cuda libraries from cuml host in recipe
dantegd Jul 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ concurrency:
jobs:
cpp-build:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@cuda-120
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -38,7 +38,7 @@ jobs:
if: github.ref_type == 'branch'
needs: [python-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@cuda-120
with:
arch: "amd64"
branch: ${{ inputs.branch }}
Expand All @@ -51,7 +51,7 @@ jobs:
python-build:
needs: [cpp-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@cuda-120
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -60,15 +60,15 @@ jobs:
upload-conda:
needs: [cpp-build, python-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@cuda-120
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
wheel-build-cuml:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@cuda-120
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -88,7 +88,7 @@ jobs:
wheel-publish-cuml:
needs: wheel-build-cuml
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-publish.yml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-publish.yml@cuda-120
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand Down
22 changes: 11 additions & 11 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,48 +23,48 @@ jobs:
- wheel-build-cuml
- wheel-tests-cuml
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@cuda-120
checks:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@cuda-120
with:
enable_check_generated_files: false
conda-cpp-build:
needs: checks
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@cuda-120
with:
build_type: pull-request
conda-cpp-tests:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@cuda-120
with:
build_type: pull-request
conda-python-build:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@cuda-120
with:
build_type: pull-request
conda-python-tests-singlegpu:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-120
with:
build_type: pull-request
test_script: "ci/test_python_singlegpu.sh"
conda-python-tests-dask:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-120
with:
build_type: pull-request
test_script: "ci/test_python_dask.sh"
conda-notebook-tests:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@cuda-120
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -74,7 +74,7 @@ jobs:
docs-build:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/custom-job.yaml@cuda-120
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -84,7 +84,7 @@ jobs:
wheel-build-cuml:
needs: checks
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-build.yml@cuda-120
with:
build_type: pull-request
package-name: cuml
Expand All @@ -96,7 +96,7 @@ jobs:
wheel-tests-cuml:
needs: wheel-build-cuml
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@cuda-120
with:
build_type: pull-request
package-name: cuml
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ on:
jobs:
conda-cpp-tests:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@cuda-120
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
conda-python-tests-singlegpu:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-120
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -33,7 +33,7 @@ jobs:
test_script: "ci/test_python_singlegpu.sh"
conda-python-tests-dask:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-120
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand All @@ -42,7 +42,7 @@ jobs:
test_script: "ci/test_python_dask.sh"
wheel-tests-cuml:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@branch-23.08
uses: rapidsai/shared-action-workflows/.github/workflows/wheels-manylinux-test.yml@cuda-120
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand Down
5 changes: 3 additions & 2 deletions conda/environments/all_cuda-118_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ channels:
dependencies:
- c-compiler
- cmake>=3.26.4
- cuda-python>=11.7.1,<12.0
- cudatoolkit=11.8
- cuda-python>=11.7.1,<12.0a0
- cuda-version=11.8
- cudatoolkit
- cudf==23.8.*
- cupy>=12.0.0
- cxx-compiler
Expand Down
77 changes: 77 additions & 0 deletions conda/environments/all_cuda-120_arch-x86_64.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# This file is generated by `rapids-dependency-file-generator`.
# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
channels:
- rapidsai
- rapidsai-nightly
- dask/label/dev
- conda-forge
- nvidia
dependencies:
- c-compiler
- cmake>=3.26.4
- cuda-cudart-dev
- cuda-nvcc
- cuda-profiler-api
- cuda-python>=12.0,<13.0a0
- cuda-version=12.0
- cudf==23.8.*
- cupy>=12.0.0
- cxx-compiler
- cython>=0.29,<0.30
- dask-core>=2023.5.1
- dask-cuda==23.8.*
- dask-cudf==23.8.*
- dask-ml
- dask>=2023.5.1
- distributed>=2023.5.1
- doxygen=1.8.20
- gcc_linux-64=11.*
- gmock>=1.13.0
- graphviz
- gtest>=1.13.0
- hdbscan
- hypothesis>=6.0,<7
- ipykernel
- ipython
- joblib>=0.11
- libcublas-dev
- libcufft-dev
- libcumlprims==23.8.*
- libcurand-dev
- libcusolver-dev
- libcusparse-dev
- libraft-headers==23.8.*
- libraft==23.8.*
- librmm==23.8.*
- nbsphinx
- ninja
- nltk
- numba>=0.57
- numpydoc
- pip
- pydata-sphinx-theme
- pylibraft==23.8.*
- pynndescent==0.5.8
- pytest
- pytest-benchmark
- pytest-cases
- pytest-cov
- pytest-xdist
- python>=3.9,<3.11
- raft-dask==23.8.*
- recommonmark
- rmm==23.8.*
- scikit-build>=0.13.1
- scikit-learn==1.2
- scipy>=1.8.0
- seaborn
- sphinx-copybutton
- sphinx-markdown-tables
- sphinx<6
- statsmodels
- sysroot_linux-64==2.17
- treelite==3.2.0
- umap-learn==0.5.3
- pip:
- git+https://github.com/dask/dask-glm@main
name: all_cuda-120_arch-x86_64
3 changes: 2 additions & 1 deletion conda/environments/cpp_all_cuda-118_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ channels:
dependencies:
- c-compiler
- cmake>=3.26.4
- cudatoolkit=11.8
- cuda-version=11.8
- cudatoolkit
- cxx-compiler
- gcc_linux-64=11.*
- gmock>=1.13.0
Expand Down
31 changes: 31 additions & 0 deletions conda/environments/cpp_all_cuda-120_arch-x86_64.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# This file is generated by `rapids-dependency-file-generator`.
# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
channels:
- rapidsai
- rapidsai-nightly
- dask/label/dev
- conda-forge
- nvidia
dependencies:
- c-compiler
- cmake>=3.26.4
- cuda-cudart-dev
- cuda-nvcc
- cuda-profiler-api
- cuda-version=12.0
- cxx-compiler
- gcc_linux-64=11.*
- gmock>=1.13.0
- gtest>=1.13.0
- libcublas-dev
- libcufft-dev
- libcumlprims==23.8.*
- libcurand-dev
- libcusolver-dev
- libcusparse-dev
- libraft-headers==23.8.*
- libraft==23.8.*
- librmm==23.8.*
- ninja
- sysroot_linux-64==2.17
name: cpp_all_cuda-120_arch-x86_64
19 changes: 3 additions & 16 deletions conda/recipes/cuml/conda_build_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ cxx_compiler_version:
- 11

cuda_compiler:
- cuda-nvcc

cuda11_compiler:
- nvcc

cmake_version:
Expand All @@ -13,21 +16,5 @@ cmake_version:
sysroot_version:
- "=2.17"

# The CTK libraries below are missing from the conda-forge::cudatoolkit
# package. The "*_host_*" version specifiers correspond to `11.8` packages and the
# "*_run_*" version specifiers correspond to `11.x` packages.

libcublas_host_version:
- "=11.11.3.6"

libcurand_host_version:
- "=10.3.0.86"

libcusolver_host_version:
- "=11.4.1.48"

libcusparse_host_version:
- "=11.7.5.86"

treelite_version:
- "=3.2.0"
Loading