Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create cugraph-equivariant package #4036

Merged
merged 27 commits into from
Jan 29, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
a859eee
move scripts over to cugraph repo
tingyu66 Dec 4, 2023
b35b307
Merge branch 'branch-24.02' into cugraph-equivariant
tingyu66 Dec 12, 2023
38d8429
Merge branch 'branch-24.02' into cugraph-equivariant
tingyu66 Dec 12, 2023
d2186ee
test conda build
tingyu66 Dec 12, 2023
148cb4e
separate conda build into a new job
tingyu66 Dec 12, 2023
15404b5
remove cpp channel
tingyu66 Dec 12, 2023
3c58d7c
fix workflow
tingyu66 Dec 12, 2023
834c1ba
build wheel
tingyu66 Dec 14, 2023
d27a42b
merge in branch-24.02
tingyu66 Jan 10, 2024
43b97b8
Merge branch 'branch-24.02' into cugraph-equivariant
tingyu66 Jan 10, 2024
1f4077a
year
tingyu66 Jan 10, 2024
6462cc3
revise mlp support
tingyu66 Jan 12, 2024
2259a8d
test with mlps
tingyu66 Jan 12, 2024
c525bef
test wheel
tingyu66 Jan 12, 2024
78fc221
Merge branch 'branch-24.02' into cugraph-equivariant
tingyu66 Jan 13, 2024
3fd56dc
absorb conda build into build_python script
tingyu66 Jan 13, 2024
924e55b
update based on cugraph-ops pr
tingyu66 Jan 18, 2024
0a9c4b8
style
tingyu66 Jan 19, 2024
9547bd5
Merge branch 'branch-24.02' into cugraph-equivariant
tingyu66 Jan 23, 2024
b9ce1ca
address comments
tingyu66 Jan 23, 2024
4457eff
mlp_activation accept sequences, imp docstrings
tingyu66 Jan 23, 2024
65ceca3
clean up
tingyu66 Jan 24, 2024
f848852
update workflow files
tingyu66 Jan 24, 2024
c0abf8c
correct test script
tingyu66 Jan 24, 2024
40b972c
Update python/cugraph-equivariant/cugraph_equivariant/tests/test_tens…
tingyu66 Jan 24, 2024
6dff47a
Update python/cugraph-equivariant/cugraph_equivariant/nn/tensor_produ…
tingyu66 Jan 24, 2024
3ee809b
format
tingyu66 Jan 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ jobs:
- wheel-tests-cugraph-dgl
- wheel-build-cugraph-pyg
- wheel-tests-cugraph-pyg
- wheel-build-cugraph-equivariant
- wheel-tests-cugraph-equivariant
- devcontainer
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
Expand Down Expand Up @@ -161,6 +163,20 @@ jobs:
build_type: pull-request
script: ci/test_wheel_cugraph-pyg.sh
matrix_filter: map(select(.ARCH == "amd64" and .CUDA_VER == "11.8.0"))
wheel-build-cugraph-equivariant:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: pull-request
script: ci/build_wheel_cugraph-equivariant.sh
wheel-tests-cugraph-equivariant:
needs: wheel-build-cugraph-equivariant
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
with:
build_type: pull-request
script: ci/test_wheel_cugraph-equivariant.sh
matrix_filter: map(select(.ARCH == "amd64"))
devcontainer:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/[email protected]
Expand Down
15 changes: 13 additions & 2 deletions build.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash

# Copyright (c) 2019-2023, NVIDIA CORPORATION.
# Copyright (c) 2019-2024, NVIDIA CORPORATION.

# cugraph build script

Expand Down Expand Up @@ -31,6 +31,7 @@ VALIDARGS="
cugraph-service
cugraph-pyg
cugraph-dgl
cugraph-equivariant
nx-cugraph
cpp-mgtests
cpp-mtmgtests
Expand Down Expand Up @@ -60,6 +61,7 @@ HELP="$0 [<target> ...] [<flag> ...]
cugraph-service - build the cugraph-service_client and cugraph-service_server Python package
cugraph-pyg - build the cugraph-pyg Python package
cugraph-dgl - build the cugraph-dgl extensions for DGL
cugraph-equivariant - build the cugraph-equivariant Python package
nx-cugraph - build the nx-cugraph Python package
cpp-mgtests - build libcugraph and libcugraph_etl MG tests. Builds MPI communicator, adding MPI as a dependency.
cpp-mtmgtests - build libcugraph MTMG tests. Adds UCX as a dependency (temporary).
Expand Down Expand Up @@ -222,7 +224,7 @@ if hasArg uninstall; then
# removes the latest one and leaves the others installed. build.sh uninstall
# can be run multiple times to remove all of them, but that is not obvious.
pip uninstall -y pylibcugraph cugraph cugraph-service-client cugraph-service-server \
cugraph-dgl cugraph-pyg nx-cugraph
cugraph-dgl cugraph-pyg cugraph-equivariant nx-cugraph
fi

if hasArg clean; then
Expand Down Expand Up @@ -359,6 +361,15 @@ if hasArg cugraph-dgl || hasArg all; then
fi
fi

# Build and install the cugraph-equivariant Python package
if hasArg cugraph-equivariant || hasArg all; then
if hasArg --clean; then
cleanPythonDir ${REPODIR}/python/cugraph-equivariant
else
python ${PYTHON_ARGS_FOR_INSTALL} ${REPODIR}/python/cugraph-equivariant
fi
fi

# Build and install the nx-cugraph Python package
if hasArg nx-cugraph || hasArg all; then
if hasArg --clean; then
Expand Down
5 changes: 5 additions & 0 deletions ci/build_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -89,4 +89,9 @@ if [[ ${RAPIDS_CUDA_MAJOR} == "11" ]]; then
conda/recipes/cugraph-dgl
fi

rapids-conda-retry mambabuild \
--no-test \
--channel "${RAPIDS_CONDA_BLD_OUTPUT_DIR}" \
conda/recipes/cugraph-equivariant

rapids-upload-conda-to-s3 python
3 changes: 2 additions & 1 deletion ci/build_wheel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ python -m pip wheel . -w dist -vvv --no-deps --disable-pip-version-check
# pure-python packages should not have auditwheel run on them.
if [[ ${package_name} == "nx-cugraph" ]] || \
[[ ${package_name} == "cugraph-dgl" ]] || \
[[ ${package_name} == "cugraph-pyg" ]]; then
[[ ${package_name} == "cugraph-pyg" ]] || \
[[ ${package_name} == "cugraph-equivariant" ]]; then
RAPIDS_PY_WHEEL_NAME="${package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 dist
else
mkdir -p final_dist
Expand Down
6 changes: 6 additions & 0 deletions ci/build_wheel_cugraph-equivariant.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/bash
# Copyright (c) 2024, NVIDIA CORPORATION.

set -euo pipefail

./ci/build_wheel.sh cugraph-equivariant python/cugraph-equivariant
34 changes: 34 additions & 0 deletions ci/test_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -247,5 +247,39 @@ else
rapids-logger "skipping cugraph_pyg pytest on CUDA != 11.8"
fi

if [[ "${RAPIDS_CUDA_VERSION}" == "11.8.0" ]]; then
if [[ "${RUNNER_ARCH}" != "ARM64" ]]; then
# Reuse cugraph-dgl's test env for cugraph-equivariant
set +u
conda activate test_cugraph_dgl
set -u
pip install e3nn==0.5.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, but would this be better off as a dependency in the py_test_cugraph_equivariant section of the dependencies.yaml file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally yes, but e3nn depends on PyTorch. Having that in pyproject.toml might pull wrong versions of pytorch for users. cugraph-dgl and -pyg's pyproject.toml does not have pytorch either (I guess for the same reason).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tingyu66 do you want me to remove pytorch from the dependencies of e3nn?


rapids-print-env

rapids-logger "pytest cugraph-equivariant"
pushd python/cugraph-equivariant/cugraph_equivariant
pytest \
--cache-clear \
--junitxml="${RAPIDS_TESTS_DIR}/junit-cugraph-dgl.xml" \
--cov-config=../../.coveragerc \
--cov=cugraph_dgl \
--cov-report=xml:"${RAPIDS_COVERAGE_DIR}/cugraph-dgl-coverage.xml" \
--cov-report=term \
.
popd

# Reactivate the test environment back
set +u
conda deactivate
conda activate test
set -u
else
rapids-logger "skipping cugraph-equivariant pytest on ARM64"
fi
else
rapids-logger "skipping cugraph-equivariant pytest on CUDA!=11.8"
fi

rapids-logger "Test script exiting with value: $EXITCODE"
exit ${EXITCODE}
33 changes: 33 additions & 0 deletions ci/test_wheel_cugraph-equivariant.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/bash
# Copyright (c) 2024, NVIDIA CORPORATION.

set -eoxu pipefail

package_name="cugraph-equivariant"
package_dir="python/cugraph-equivariant"

python_package_name=$(echo ${package_name}|sed 's/-/_/g')

mkdir -p ./dist
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

# use 'ls' to expand wildcard before adding `[extra]` requires for pip
RAPIDS_PY_WHEEL_NAME="${package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 ./dist
# pip creates wheels using python package names
python -m pip install $(ls ./dist/${python_package_name}*.whl)[test]


PKG_CUDA_VER="$(echo ${CUDA_VERSION} | cut -d '.' -f1,2 | tr -d '.')"
PKG_CUDA_VER_MAJOR=${PKG_CUDA_VER:0:2}
if [[ "${PKG_CUDA_VER_MAJOR}" == "12" ]]; then
PYTORCH_CUDA_VER="121"
else
PYTORCH_CUDA_VER=$PKG_CUDA_VER
fi
PYTORCH_URL="https://download.pytorch.org/whl/cu${PYTORCH_CUDA_VER}"

rapids-logger "Installing PyTorch and e3nn"
rapids-retry python -m pip install torch --index-url ${PYTORCH_URL}
rapids-retry python -m pip install e3nn

python -m pytest python/cugraph-equivariant/cugraph_equivariant/tests
7 changes: 7 additions & 0 deletions conda/recipes/cugraph-equivariant/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

# Copyright (c) 2024, NVIDIA CORPORATION.

# This assumes the script is executed from the root of the repo directory

./build.sh cugraph-equivariant
37 changes: 37 additions & 0 deletions conda/recipes/cugraph-equivariant/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Copyright (c) 2024, NVIDIA CORPORATION.

{% set version = environ['RAPIDS_PACKAGE_VERSION'].lstrip('v') + environ.get('VERSION_SUFFIX', '') %}
{% set minor_version = version.split('.')[0] + '.' + version.split('.')[1] %}
{% set py_version = environ['CONDA_PY'] %}
{% set date_string = environ['RAPIDS_DATE_STRING'] %}

package:
name: cugraph-equivariant
version: {{ version }}

source:
path: ../../..

build:
number: {{ GIT_DESCRIBE_NUMBER }}
build:
number: {{ GIT_DESCRIBE_NUMBER }}
string: py{{ py_version }}_{{ date_string }}_{{ GIT_DESCRIBE_HASH }}_{{ GIT_DESCRIBE_NUMBER }}

requirements:
host:
- python
run:
- pylibcugraphops ={{ minor_version }}
- python

tests:
imports:
- cugraph_equivariant

about:
home: https://rapids.ai/
dev_url: https://github.com/rapidsai/cugraph
license: Apache-2.0
license_file: ../../../LICENSE
summary: GPU-accelerated equivariant convolutional layers.
22 changes: 22 additions & 0 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,28 @@ files:
key: test
includes:
- test_python_common
py_build_cugraph_equivariant:
output: pyproject
pyproject_dir: python/cugraph-equivariant
extras:
table: build-system
includes:
- python_build_wheel
py_run_cugraph_equivariant:
output: pyproject
pyproject_dir: python/cugraph-equivariant
extras:
table: project
includes:
- depends_on_pylibcugraphops
py_test_cugraph_equivariant:
output: pyproject
pyproject_dir: python/cugraph-equivariant
extras:
table: project.optional-dependencies
key: test
includes:
- test_python_common
py_build_cugraph_service_client:
output: pyproject
pyproject_dir: python/cugraph-service/client
Expand Down
1 change: 1 addition & 0 deletions python/cugraph-equivariant/LICENSE
16 changes: 16 additions & 0 deletions python/cugraph-equivariant/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# cugraph-equivariant

## Description

cugraph-equivariant library provides fast symmetry-preserving (equivariant) operations and convolutional layers, to accelerate the equivariant neural networks in drug discovery and other domains.

## Build from source

Developers are suggested to create a conda environment that includes the runtime and test dependencies and pip install `cugraph-equivariant` in an editable mode.

```bash
# for cuda 11.8
mamba env create -n cugraph_equivariant -f python/cugraph-equivariant/conda/cugraph_equivariant_dev_cuda-118_arch-x86_64.yaml
conda activate cugraph_equivariant
./build_component.sh -n cugraph-equivariant
```
1 change: 1 addition & 0 deletions python/cugraph-equivariant/cugraph_equivariant/VERSION
14 changes: 14 additions & 0 deletions python/cugraph-equivariant/cugraph_equivariant/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright (c) 2024, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from cugraph_equivariant._version import __git_commit__, __version__
27 changes: 27 additions & 0 deletions python/cugraph-equivariant/cugraph_equivariant/_version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Copyright (c) 2024, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import importlib.resources

# Read VERSION file from the module that is symlinked to VERSION file
# in the root of the repo at build time or copied to the module at
# installation. VERSION is a separate file that allows CI build-time scripts
# to update version info (including commit hashes) without modifying
# source files.
__version__ = (
importlib.resources.files("cugraph_equivariant")
.joinpath("VERSION")
.read_text()
.strip()
)
__git_commit__ = ""
21 changes: 21 additions & 0 deletions python/cugraph-equivariant/cugraph_equivariant/nn/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Copyright (c) 2024, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from .tensor_product_conv import FullyConnectedTensorProductConv

DiffDockTensorProductConv = FullyConnectedTensorProductConv
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this alias?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is from one of our discussions on Monday, but it's totally optional. @mariogeiger Do you think we need the alias?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for me, the name FullyConnectedTensorProductConv is perfect by its own


__all__ = [
"FullyConnectedTensorProductConv",
"DiffDockTensorProductConv",
]
Loading
Loading