Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update builds for CUDA 11.8 and Python 3.10 #1120

Merged
merged 14 commits into from
Jan 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ concurrency:
jobs:
cpp-build:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@cuda-118
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -37,7 +37,7 @@ jobs:
python-build:
needs: [cpp-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@cuda-118
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -46,7 +46,7 @@ jobs:
upload-conda:
needs: [cpp-build, python-build]
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-upload-packages.yaml@cuda-118
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,33 +22,33 @@ jobs:
- wheel-build-raft-dask
- wheel-tests-raft-dask
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/pr-builder.yaml@cuda-118
checks:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/checks.yaml@cuda-118
conda-cpp-build:
needs: checks
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@cuda-118
with:
build_type: pull-request
node_type: cpu16
conda-cpp-tests:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@cuda-118
with:
build_type: pull-request
conda-python-build:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-matrix-build.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-build.yaml@cuda-118
with:
build_type: pull-request
conda-python-tests:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-118
with:
build_type: pull-request
wheel-build-pylibraft:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ on:
jobs:
conda-cpp-tests:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests.yaml@cuda-118
with:
build_type: nightly
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
conda-python-tests:
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@main
uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-tests.yaml@cuda-118
with:
build_type: nightly
branch: ${{ inputs.branch }}
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ Several CMake targets can be made available by adding components in the table be
The easiest way to build RAFT from source is to use the `build.sh` script at the root of the repository:
1. Create an environment with the needed dependencies:
```
mamba env create --name raft_dev_env -f conda/environments/all_cuda-115_arch-x86_64.yaml
mamba env create --name raft_dev_env -f conda/environments/all_cuda-118_arch-x86_64.yaml
mamba activate raft_dev_env
```
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,37 @@ dependencies:
- clang-tools=11.1.0
- clang=11.1.0
- cmake>=3.23.1,!=3.25.0
- cuda-profiler-api>=11.4.240,<=11.8.86
- cuda-profiler-api=11.8.86
- cuda-python >=11.7.1,<12.0
- cudatoolkit=11.5
- cudatoolkit=11.8
- cupy
- cxx-compiler
- cython>=0.29,<0.30
- dask-cuda=23.02.*
- dask-cuda=23.02
- dask>=2022.12.0
- distributed>=2022.12.0
- doxygen>=1.8.20
- faiss-proc=*=cuda
- gcc_linux-64=9.*
- libcublas-dev>=11.7.3.1,<=11.7.4.6
- libcublas>=11.7.3.1,<=11.7.4.6
- libcurand-dev>=10.2.6.48,<=10.2.7.107
- libcurand>=10.2.6.48,<=10.2.7.107
- libcusolver-dev>=11.2.1.48,<=11.3.2.107
- libcusolver>=11.2.1.48,<=11.3.2.107
- libcusparse-dev>=11.7.0.31,<=11.7.0.107
- libcusparse>=11.7.0.31,<=11.7.0.107
- libfaiss>=1.7.0=cuda*
- gcc_linux-64=9
- libcublas-dev=11.11.3.6
- libcublas=11.11.3.6
- libcurand-dev=10.3.0.86
- libcurand=10.3.0.86
- libcusolver-dev=11.4.1.48
- libcusolver=11.4.1.48
- libcusparse-dev=11.7.5.86
- libcusparse=11.7.5.86
- libfaiss>=1.7.1=cuda*
- ninja
- pytest
- pytest-cov
- rmm=23.02.*
- rmm=23.02
- scikit-build>=0.13.1
- scikit-learn
- scipy
- sphinx-markdown-tables
- sysroot_linux-64==2.17
- ucx-proc=*=gpu
- ucx-py=0.30.*
- ucx-py=0.30
- ucx>=1.13.0
name: all_cuda-115_arch-x86_64
name: all_cuda-118_arch-x86_64
22 changes: 11 additions & 11 deletions conda/recipes/libraft/conda_build_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,42 +20,42 @@ gtest_version:
- "=1.10.0"

libfaiss_version:
- "1.7.0 *_cuda"
- "1.7.2 *_cuda"

# The CTK libraries below are missing from the conda-forge::cudatoolkit
# package. The "*_host_*" version specifiers correspond to `11.5` packages and the
# package. The "*_host_*" version specifiers correspond to `11.8` packages and the
# "*_run_*" version specifiers correspond to `11.x` packages.

libcublas_host_version:
- ">=11.7.3.1,<=11.7.4.6"
- "=11.11.3.6"

libcublas_run_version:
- ">=11.5.2.43,<=11.11.3.6"
- ">=11.5.2.43,<12.0.0"

libcurand_host_version:
- ">=10.2.6.48,<=10.2.7.107"
- "=10.3.0.86"

libcurand_run_version:
- ">=10.2.5.43,<=10.3.0.86"
- ">=10.2.5.43,<10.3.1"

libcusolver_host_version:
- ">=11.2.1.48,<=11.3.2.107"
- "=11.4.1.48"

libcusolver_run_version:
- ">=11.2.0.43,<=11.4.1.48"
- ">=11.2.0.43,<11.4.2"

libcusparse_host_version:
- ">=11.7.0.31,<=11.7.0.107"
- "=11.7.5.86"

libcusparse_run_version:
- ">=11.6.0.43,<=11.7.5.86"
- ">=11.6.0.43,<12.0.0"

# `cuda-profiler-api` only has `11.8.0` and `12.0.0` packages for all
# architectures. The "*_host_*" version specifiers correspond to `11.8` packages and the
# "*_run_*" version specifiers correspond to `11.x` packages.

cuda_profiler_api_host_version:
- ">=11.8.86,<12"
- "=11.8.86"

cuda_profiler_api_run_version:
- ">=11.4.240,<12"
33 changes: 25 additions & 8 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ files:
all:
output: conda
matrix:
cuda: ["11.5"]
cuda: ["11.8"]
arch: [x86_64]
includes:
- build
Expand Down Expand Up @@ -53,12 +53,12 @@ dependencies:
- matrix:
arch: x86_64
packages:
- gcc_linux-64=9.*
- gcc_linux-64=9
- sysroot_linux-64==2.17
- matrix:
arch: aarch64
packages:
- gcc_linux-aarch64=9.*
- gcc_linux-aarch64=9
- sysroot_linux-aarch64==2.17
checks:
common:
Expand All @@ -77,6 +77,19 @@ dependencies:
specific:
- output_types: conda
matrices:
- matrix:
cuda: "11.8"
packages:
- cudatoolkit=11.8
- cuda-profiler-api=11.8.86
- libcublas-dev=11.11.3.6
- libcublas=11.11.3.6
- libcurand-dev=10.3.0.86
- libcurand=10.3.0.86
- libcusolver-dev=11.4.1.48
- libcusolver=11.4.1.48
- libcusparse-dev=11.7.5.86
- libcusparse=11.7.5.86
- matrix:
cuda: "11.5"
packages:
Expand Down Expand Up @@ -142,22 +155,26 @@ dependencies:
py: "3.9"
packages:
- python=3.9
- matrix:
py: "3.10"
packages:
- python=3.10
- matrix:
packages:
- python>=3.8,<3.10
- python>=3.8,<3.11
run:
common:
- output_types: [conda]
packages:
- rmm=23.02.*
- rmm=23.02
- dask>=2022.12.0
- distributed>=2022.12.0
- ucx>=1.13.0
- ucx-py=0.30.*
- ucx-py=0.30
- ucx-proc=*=gpu
- libfaiss>=1.7.0=cuda*
- libfaiss>=1.7.1=cuda*
- faiss-proc=*=cuda
- dask-cuda=23.02.*
- dask-cuda=23.02
test_python:
common:
- output_types: [conda, requirements]
Expand Down
4 changes: 2 additions & 2 deletions docs/source/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,10 +180,10 @@ Currently, shared libraries are provided for the `libraft-nn` and `libraft-dista

### Python

Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. In addition you will have to manually install `nvcc` as it will not be installed as part of the conda environment. The following example will install create and install dependencies for a CUDA 11.5 conda environment:
Conda environment scripts are provided for installing the necessary dependencies for building and using the Python APIs. It is preferred to use `mamba`, as it provides significant speedup over `conda`. In addition you will have to manually install `nvcc` as it will not be installed as part of the conda environment. The following example will install create and install dependencies for a CUDA 11.8 conda environment:

```bash
mamba env create --name raft_env_name -f conda/environments/all_cuda-115_arch-x86_64.yaml
mamba env create --name raft_env_name -f conda/environments/all_cuda-118_arch-x86_64.yaml
mamba activate raft_env_name
```

Expand Down
1 change: 1 addition & 0 deletions python/pylibraft/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ def get_versions():
"Programming Language :: Python",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
],
author="NVIDIA Corporation",
include_package_data=True,
Expand Down
1 change: 1 addition & 0 deletions python/raft-dask/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def get_versions():
"Programming Language :: Python",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
],
author="NVIDIA Corporation",
include_package_data=True,
Expand Down