Skip to content

Commit

Permalink
Update builds for CUDA 11.8 and Python 3.10 (#1120)
Browse files Browse the repository at this point in the history 
This PR updates the `raft` CI workflows to build against the CUDA `11.8` / Python `3.10` [branch](https://github.com/rapidsai/shared-action-workflows/tree/cuda-118) of the `shared-action-workflows` repository.

Authors:
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Ray Douglass (https://github.com/raydouglass)
  - Jake Awe (https://github.com/AyodeAwe)
  - Corey J. Nolet (https://github.com/cjnolet)

URL: #1120
  • Loading branch information
@ajschmidt8
ajschmidt8 authored Jan 17, 2023
1 parent efd42c9 commit 9a97b52
Show file tree
Hide file tree
Showing 10 changed files with 68 additions and 49 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/build.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ concurrency:
jobs:
cpp-build:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@cuda-118
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -37,7 +37,7 @@ jobs:
python-build:
needs: [cpp-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@cuda-118
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -46,7 +46,7 @@ jobs:
upload-conda:
needs: [cpp-build, python-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@cuda-118
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/pr.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,33 +22,33 @@ jobs:
- wheel-build-raft-dask
- wheel-tests-raft-dask
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@cuda-118
checks:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@cuda-118
conda-cpp-build:
needs: checks
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@cuda-118
with:
build_type: pull-request
node_type: cpu16
conda-cpp-tests:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@cuda-118
with:
build_type: pull-request
conda-python-build:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@cuda-118
with:
build_type: pull-request
conda-python-tests:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-118
with:
build_type: pull-request
wheel-build-pylibraft:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ on:
jobs:
conda-cpp-tests:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@cuda-118
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
conda-python-tests:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-118
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ Several CMake targets can be made available by adding components in the table be
The easiest way to build RAFT from source is to use the `build.sh` script at the root of the repository:
1. Create an environment with the needed dependencies:
```
mamba env create --name raft_dev_env -f conda/environments/all_cuda-115_arch-x86_64.yaml
mamba env create --name raft_dev_env -f conda/environments/all_cuda-118_arch-x86_64.yaml
mamba activate raft_dev_env
```
```
Expand Down
32 changes: 16 additions & 16 deletions ...nvironments/all_cuda-115_arch-x86_64.yaml → ...nvironments/all_cuda-118_arch-x86_64.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,37 @@ dependencies:
- clang-tools=11.1.0
- clang=11.1.0
- cmake>=3.23.1,!=3.25.0
- cuda-profiler-api>=11.4.240,<=11.8.86
- cuda-profiler-api=11.8.86
- cuda-python >=11.7.1,<12.0
- cudatoolkit=11.5
- cudatoolkit=11.8
- cupy
- cxx-compiler
- cython>=0.29,<0.30
- dask-cuda=23.02.*
- dask-cuda=23.02
- dask>=2022.12.0
- distributed>=2022.12.0
- doxygen>=1.8.20
- faiss-proc=*=cuda
- gcc_linux-64=9.*
- libcublas-dev>=11.7.3.1,<=11.7.4.6
- libcublas>=11.7.3.1,<=11.7.4.6
- libcurand-dev>=10.2.6.48,<=10.2.7.107
- libcurand>=10.2.6.48,<=10.2.7.107
- libcusolver-dev>=11.2.1.48,<=11.3.2.107
- libcusolver>=11.2.1.48,<=11.3.2.107
- libcusparse-dev>=11.7.0.31,<=11.7.0.107
- libcusparse>=11.7.0.31,<=11.7.0.107
- libfaiss>=1.7.0=cuda*
- gcc_linux-64=9
- libcublas-dev=11.11.3.6
- libcublas=11.11.3.6
- libcurand-dev=10.3.0.86
- libcurand=10.3.0.86
- libcusolver-dev=11.4.1.48
- libcusolver=11.4.1.48
- libcusparse-dev=11.7.5.86
- libcusparse=11.7.5.86
- libfaiss>=1.7.1=cuda*
- ninja
- pytest
- pytest-cov
- rmm=23.02.*
- rmm=23.02
- scikit-build>=0.13.1
- scikit-learn
- scipy
- sphinx-markdown-tables
- sysroot_linux-64==2.17
- ucx-proc=*=gpu
- ucx-py=0.30.*
- ucx-py=0.30
- ucx>=1.13.0
name: all_cuda-115_arch-x86_64
name: all_cuda-118_arch-x86_64
22 changes: 11 additions & 11 deletions conda/recipes/libraft/conda_build_config.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,42 +20,42 @@ gtest_version:
- "=1.10.0"

libfaiss_version:
- "1.7.0 *_cuda"
- "1.7.2 *_cuda"

# The CTK libraries below are missing from the conda-forge::cudatoolkit
# package. The "*_host_*" version specifiers correspond to `11.5` packages and the
# package. The "*_host_*" version specifiers correspond to `11.8` packages and the
# "*_run_*" version specifiers correspond to `11.x` packages.

libcublas_host_version:
- ">=11.7.3.1,<=11.7.4.6"
- "=11.11.3.6"

libcublas_run_version:
- ">=11.5.2.43,<=11.11.3.6"
- ">=11.5.2.43,<12.0.0"

libcurand_host_version:
- ">=10.2.6.48,<=10.2.7.107"
- "=10.3.0.86"

libcurand_run_version:
- ">=10.2.5.43,<=10.3.0.86"
- ">=10.2.5.43,<10.3.1"

libcusolver_host_version:
- ">=11.2.1.48,<=11.3.2.107"
- "=11.4.1.48"

libcusolver_run_version:
- ">=11.2.0.43,<=11.4.1.48"
- ">=11.2.0.43,<11.4.2"

libcusparse_host_version:
- ">=11.7.0.31,<=11.7.0.107"
- "=11.7.5.86"

libcusparse_run_version:
- ">=11.6.0.43,<=11.7.5.86"
- ">=11.6.0.43,<12.0.0"

# `cuda-profiler-api` only has `11.8.0` and `12.0.0` packages for all
# architectures. The "*_host_*" version specifiers correspond to `11.8` packages and the
# "*_run_*" version specifiers correspond to `11.x` packages.

cuda_profiler_api_host_version:
- ">=11.8.86,<12"
- "=11.8.86"

cuda_profiler_api_run_version:
- ">=11.4.240,<12"
33 changes: 25 additions & 8 deletions dependencies.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ files:
all:
output: conda
matrix:
cuda: ["11.5"]
cuda: ["11.8"]
arch: [x86_64]
includes:
- build
Expand Down Expand Up @@ -53,12 +53,12 @@ dependencies:
- matrix:
arch: x86_64
packages:
- gcc_linux-64=9.*
- gcc_linux-64=9
- sysroot_linux-64==2.17
- matrix:
arch: aarch64
packages:
- gcc_linux-aarch64=9.*
- gcc_linux-aarch64=9
- sysroot_linux-aarch64==2.17
checks:
common:
Expand All @@ -77,6 +77,19 @@ dependencies:
specific:
- output_types: conda
matrices:
- matrix:
cuda: "11.8"
packages:
- cudatoolkit=11.8
- cuda-profiler-api=11.8.86
- libcublas-dev=11.11.3.6
- libcublas=11.11.3.6
- libcurand-dev=10.3.0.86
- libcurand=10.3.0.86
- libcusolver-dev=11.4.1.48
- libcusolver=11.4.1.48
- libcusparse-dev=11.7.5.86
- libcusparse=11.7.5.86
- matrix:
cuda: "11.5"
packages:
Expand Down Expand Up @@ -142,22 +155,26 @@ dependencies:
py: "3.9"
packages:
- python=3.9
- matrix:
py: "3.10"
packages:
- python=3.10
- matrix:
packages:
- python>=3.8,<3.10
- python>=3.8,<3.11
run:
common:
- output_types: [conda]
packages:
- rmm=23.02.*
- rmm=23.02
- dask>=2022.12.0
- distributed>=2022.12.0
- ucx>=1.13.0
- ucx-py=0.30.*
- ucx-py=0.30
- ucx-proc=*=gpu
- libfaiss>=1.7.0=cuda*
- libfaiss>=1.7.1=cuda*
- faiss-proc=*=cuda
- dask-cuda=23.02.*
- dask-cuda=23.02
test_python:
common:
- output_types: [conda, requirements]
Expand Down
4 changes: 2 additions & 2 deletions docs/source/build.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -180,10 +180,10 @@ Currently, shared libraries are provided for the `libraft-nn` and `libraft-dista

### Python

Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. In addition you will have to manually install `nvcc` as it will not be installed as part of the conda environment. The following example will install create and install dependencies for a CUDA 11.5 conda environment:
Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. In addition you will have to manually install `nvcc` as it will not be installed as part of the conda environment. The following example will install create and install dependencies for a CUDA 11.8 conda environment:

```bash
mamba env create --name raft_env_name -f conda/environments/all_cuda-115_arch-x86_64.yaml
mamba env create --name raft_env_name -f conda/environments/all_cuda-118_arch-x86_64.yaml
mamba activate raft_env_name
```

Expand Down
1 change: 1 addition & 0 deletions python/pylibraft/setup.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ def get_versions():
"Programming Language :: Python",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
],
author="NVIDIA Corporation",
include_package_data=True,
Expand Down
1 change: 1 addition & 0 deletions python/raft-dask/setup.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def get_versions():
"Programming Language :: Python",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
],
author="NVIDIA Corporation",
include_package_data=True,
Expand Down

0 comments on commit 9a97b52

Please sign in to comment.