Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use rapids-build-backend #736

Merged
merged 3 commits into from
Jun 10, 2024
Merged

use rapids-build-backend #736

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1bada91
use rapids-build-backend
jameslamb Jun 4, 2024
2002e53
Merge branch 'branch-24.08' into rapids-build-backend
jameslamb Jun 5, 2024
eaad9f0
Merge branch 'branch-24.08' into rapids-build-backend
jameslamb Jun 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ repos:
additional_dependencies:
- tomli
- repo: https://github.com/rapidsai/dependency-file-generator
rev: v1.8.0
rev: v1.13.11
hooks:
- id: rapids-dependency-file-generator
args: ["--clean"]
4 changes: 1 addition & 3 deletions ci/build_cpp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,10 @@ export CMAKE_GENERATOR=Ninja

rapids-print-env

version=$(rapids-generate-version)

rapids-logger "Begin cpp build"

conda config --set path_conflict prevent

RAPIDS_PACKAGE_VERSION=${version} rapids-conda-retry mambabuild conda/recipes/libcucim
RAPIDS_PACKAGE_VERSION=$(rapids-generate-version) rapids-conda-retry mambabuild conda/recipes/libcucim

rapids-upload-conda-to-s3 cpp
2 changes: 1 addition & 1 deletion ci/build_docs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ rapids-logger "Create test conda environment"

rapids-dependency-file-generator \
--output conda \
--file_key docs \
--file-key docs \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee env.yaml

rapids-mamba-retry env create --yes -f env.yaml -n docs
Expand Down
13 changes: 2 additions & 11 deletions ci/build_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,7 @@ export CMAKE_GENERATOR=Ninja

rapids-print-env

package_name="cucim"
package_dir="python/cucim"
package_src_dir="${package_dir}/src/${package_name}"

version=$(rapids-generate-version)

commit=$(git rev-parse HEAD)

echo "${version}" > VERSION
sed -i "/^__git_commit__/ s/= .*/= \"${commit}\"/g" "${package_src_dir}/_version.py"
rapids-generate-version > ./VERSION

rapids-logger "Begin py build"
conda config --set path_conflict prevent
Expand All @@ -31,7 +22,7 @@ CPP_CHANNEL=$(rapids-download-conda-from-s3 cpp)

# TODO: Remove `--no-test` flag once importing on a CPU
# node works correctly
RAPIDS_PACKAGE_VERSION=${version} rapids-conda-retry mambabuild \
RAPIDS_PACKAGE_VERSION=$(head -1 ./VERSION) rapids-conda-retry mambabuild \
--no-test \
--channel "${CPP_CHANNEL}" \
conda/recipes/cucim
Expand Down
17 changes: 2 additions & 15 deletions ci/build_wheel.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,28 +12,15 @@ CMAKE_BUILD_TYPE="release"
source rapids-configure-sccache
source rapids-date-string

version=$(rapids-generate-version)
commit=$(git rev-parse HEAD)
rapids-generate-version > ./VERSION

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

# Patch project metadata files to include the CUDA version suffix and version override.
pyproject_file="${package_dir}/pyproject.toml"

PACKAGE_CUDA_SUFFIX="-${RAPIDS_PY_CUDA_SUFFIX}"
# update package name to have the cuda suffix
sed -i "s/name = \"${package_name}\"/name = \"${package_name}${PACKAGE_CUDA_SUFFIX}\"/g" ${pyproject_file}
echo "${version}" > VERSION
sed -i "/^__git_commit__/ s/= .*/= \"${commit}\"/g" "${package_src_dir}/_version.py"

if [[ ${PACKAGE_CUDA_SUFFIX} == "-cu12" ]]; then
# change pyproject.toml to use CUDA 12.x version of cupy
sed -i "s/cupy-cuda11x/cupy-cuda12x/g" ${pyproject_file}
fi

# Install pip build dependencies (not yet using pyproject.toml)
rapids-dependency-file-generator \
--file_key "py_build" \
--file-key "py_build" \
--output "requirements" \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee build_requirements.txt
pip install -r build_requirements.txt
Expand Down
2 changes: 1 addition & 1 deletion ci/check_style.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ rapids-logger "Create checks conda environment"

rapids-dependency-file-generator \
--output conda \
--file_key checks \
--file-key checks \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee env.yaml

rapids-mamba-retry env create --yes -f env.yaml -n checks
Expand Down
2 changes: 1 addition & 1 deletion ci/test_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ RAPIDS_VERSION_NUMBER=$(rapids-generate-version)
rapids-logger "Generate Python testing dependencies"
rapids-dependency-file-generator \
--output conda \
--file_key test_python \
--file-key test_python \
--matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee env.yaml

rapids-mamba-retry env create --yes -f env.yaml -n test
Expand Down
2 changes: 1 addition & 1 deletion conda/recipes/cucim/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ cp -P python/install/lib/* python/cucim/src/cucim/clara/
pushd python/cucim

echo "PYTHON: ${PYTHON}"
$PYTHON -m pip install . -vv
$PYTHON -m pip install --config-settings rapidsai.disable-cuda=true . -vv

popd
2 changes: 2 additions & 0 deletions conda/recipes/cucim/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,10 @@ requirements:
- libcucim ={{ version }}
- numpy 1.23
- python
- rapids-build-backend >=0.3.0,<0.4.0.dev0
- scikit-image >=0.19.0,<0.23.0a0
- scipy >=1.6
- setuptools >=24.2.0
run:
- {{ pin_compatible('cuda-version', max_pin='x', min_pin='x') }}
{% if cuda_major != "11" %}
Expand Down
28 changes: 25 additions & 3 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ files:
pyproject_dir: python/cucim
extras:
table: build-system
includes:
- rapids_build_setuptools
py_rapids_build:
output: pyproject
pyproject_dir: python/cucim
extras:
table: tool.rapids-build-backend
key: requires
includes:
- build
py_run:
Expand Down Expand Up @@ -92,7 +100,6 @@ dependencies:
- cxx-compiler
- output_types: [requirements, pyproject]
packages:
- setuptools>=24.2.0
- wheel
specific:
- output_types: conda
Expand Down Expand Up @@ -240,6 +247,12 @@ dependencies:
- matrix:
packages:
- python>=3.8,<3.12
rapids_build_setuptools:
common:
- output_types: [conda, requirements, pyproject]
packages:
- rapids-build-backend>=0.3.0,<0.4.0.dev0
- setuptools>=24.2.0
run:
common:
- output_types: [conda, requirements, pyproject]
Expand All @@ -266,9 +279,18 @@ dependencies:
# Not sure where these go, if anywhere:
# - openslide
# - xorg-libxcb
specific:
- output_types: [requirements, pyproject]
packages:
- cupy-cuda11x>=12.0.0
matrices:
- matrix: {cuda: "12.*"}
packages:
- cupy-cuda12x>=12.0.0
- matrix: {cuda: "11.*"}
packages:
- &cupy_cu11 cupy-cuda11x>=12.0.0
- matrix:
packages:
- *cupy_cu11
Comment on lines +290 to +293
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfamiliar with the notation, curious what the & and * in the package names do here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are YAML anchors. We defined an anchor with & and reused its value with *.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh sure, happy to explain! This is a YAML "alias", pointing to a YAML "anchor". When this document is loaded, *cupy_cu11 will be replaced with whatever value is tagged with &cupy_cu11. It's a way to reduce a bit of duplication, to keep things that should be consistent consistent.

For example, this:

limits:
  cpu: 1.0
requests:
  cpu: 1.0

Is identical to this:

limits:
  cpu: &cpu_val 1.0
requests:
  cpu: *cpu_val

For reference, see:

test_python:
common:
- output_types: [conda, requirements, pyproject]
Expand Down
14 changes: 12 additions & 2 deletions python/cucim/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
# See file LICENSE for terms.

[build-system]
build-backend = "setuptools.build_meta"
build-backend = "rapids_build_backend.build"
requires = [
"rapids-build-backend>=0.3.0,<0.4.0.dev0",
"setuptools>=24.2.0",
"wheel",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

[project]
Expand Down Expand Up @@ -90,6 +90,16 @@ docs = [
[project.entry-points."console_scripts"]
cucim = "cucim.clara.cli:main"

[tool.rapids-build-backend]
build-backend = "setuptools.build_meta"
commit-files = [
"src/cucim/COMMIT_FILE"
]
dependencies-file = "../../dependencies.yaml"
requires = [
"wheel",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

[tool.setuptools]
license-files = [
"LICENSE",
Expand Down
17 changes: 15 additions & 2 deletions python/cucim/src/cucim/_version.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,19 @@
import importlib.resources

__version__ = (
importlib.resources.files("cucim").joinpath("VERSION").read_text().strip()
importlib.resources.files(__package__)
.joinpath("VERSION")
.read_text()
.strip()
)
__git_commit__ = ""
try:
__git_commit__ = (
importlib.resources.files(__package__)
.joinpath("GIT_COMMIT")
.read_text()
.strip()
)
except FileNotFoundError:
__git_commit__ = ""

__all__ = ["__git_commit__", "__version__"]
25 changes: 25 additions & 0 deletions python/cucim/tests/unit/test_version.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#
# Copyright (c) 2024, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

import cucim


def test_version_constants_are_populated():
# __git_commit__ will only be non-empty in a built distribution
assert isinstance(cucim.__git_commit__, str)

# __version__ should always be non-empty
assert isinstance(cucim.__version__, str)
assert len(cucim.__version__) > 0