Add CI scripts to build conda packages and run tests #18
Conversation
pentschev
force-pushed
the
ci-support
branch
8 times, most recently
from
cef864e to
03be322
Compare
This was removed previously as it's not _always_ necessary, for example InfiniBand will work correctly with or without it, but the server may close too soon when using TCP.
The tests marked may hang until they timeout under certain circumstances. See rapidsai#19 .
pentschev
force-pushed
the
ci-support
branch
2 times, most recently
from
6922ddd to
bda7f2a
Compare
this will remove the duplicate Python builds that are currently resulting from using both `conda_build_config.yaml` build variants and the GHAs matrix builds
| rmm_version: | ||
| - "23.04.*" |
There was a problem hiding this comment.
Does this get updated by the ci/release/update-version.sh script? If not, should it be?
There was a problem hiding this comment.
Thanks John, done in 3707568 .
| branches: | ||
| - "branch-*" | ||
| tags: | ||
| - v[0-9].[0-9][0-9].[0-9][0-9] |
There was a problem hiding this comment.
I am guessing the major component is fixed as one digit because this aims to follow ucx versioning? Like 0.30.0? But then is the patch version also one digit or is it two digits? Like 0.30.00? Not sure what regex syntax is allowed here but I think there are limitations.
There was a problem hiding this comment.
We're doing here exactly what UCX-Py is doing, I believe this is right as it has been for a while.
dependencies.yaml
Outdated
| specific: | ||
| - output_types: conda | ||
| matrices: | ||
| - matrix: | ||
| cuda: "11.8" | ||
| packages: | ||
| - cuda-sanitizer-api=11.8.86 | ||
| - matrix: | ||
| packages: |
There was a problem hiding this comment.
Where is cuda-sanitizer-api being used? I searched briefly but didn't see it. If it's not used, the entire specific section can be deleted.
| specific: | |
| - output_types: conda | |
| matrices: | |
| - matrix: | |
| cuda: "11.8" | |
| packages: | |
| - cuda-sanitizer-api=11.8.86 | |
| - matrix: | |
| packages: |
dependencies.yaml
Outdated
| - checks | ||
| - cudatoolkit | ||
| - py_version | ||
| - run |
There was a problem hiding this comment.
| - run | |
| - run_python |
dependencies.yaml
Outdated
| - matrix: | ||
| packages: | ||
| - python>=3.8,<3.11 | ||
| run: |
There was a problem hiding this comment.
| run: | |
| run_python: |
dependencies.yaml
Outdated
| - &gmock gmock==1.10.0.* | ||
| - spdlog>=1.11.0,<1.12 | ||
| - cython>=0.29,<0.30 | ||
| - numpy>=1.21 |
There was a problem hiding this comment.
Should we use an anchor here and re-use this pinning version in the other places numpy is referenced? See >est / *gtest for an example.
dependencies.yaml
Outdated
| - python>=3.8,<3.11 | ||
| run: | ||
| common: | ||
| - output_types: [conda, requirements] |
There was a problem hiding this comment.
Some additional work is probably needed to make this file compatible with generating the build/run/test requirements in pyproject.toml. Happy to defer that to a later PR if we can file an issue.
| - output_types: [conda, requirements] | |
| - output_types: [conda, requirements, pyproject] |
ci/release/update-version.sh
Outdated
There was a problem hiding this comment.
This file also needs to update rmm_version in conda/recipes/ucxx/conda_build_config.yaml.
ci/check_style.sh
Outdated
| conda activate checks | ||
|
|
||
| # Run pre-commit checks | ||
| pre-commit run --hook-stage manual --all-files --show-diff-on-failure |
There was a problem hiding this comment.
| pre-commit run --hook-stage manual --all-files --show-diff-on-failure | |
| pre-commit run --all-files --show-diff-on-failure |
.pre-commit-config.yaml
Outdated
There was a problem hiding this comment.
I'd recommend updating the hook versions in this file with pre-commit autoupdate. Since it's a new package, that should be a better place to start. I am going to be updating hook versions in other RAPIDS repos in the next month or so.
There was a problem hiding this comment.
I don't mind if we do it in a follow-up PR, but I'd rather have something functional first and iterate on improvements later.
.github/workflows/test.yaml
Outdated
| jobs: | ||
| conda-cpp-tests: | ||
| secrets: inherit | ||
| uses: rapidsai/shared-action-workflows/.github/workflows/[email protected] |
There was a problem hiding this comment.
This seems wrong.
| uses: rapidsai/shared-action-workflows/.github/workflows/conda-python-[email protected] | |
| uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-tests[email protected] |
There was a problem hiding this comment.
Thanks @jakirkham and @bdice , I've addressed the suggestions
| branches: | ||
| - "branch-*" | ||
| tags: | ||
| - v[0-9].[0-9][0-9].[0-9][0-9] |
There was a problem hiding this comment.
We're doing here exactly what UCX-Py is doing, I believe this is right as it has been for a while.
.github/workflows/test.yaml
Outdated
| jobs: | ||
| conda-cpp-tests: | ||
| secrets: inherit | ||
| uses: rapidsai/shared-action-workflows/.github/workflows/[email protected] |
.pre-commit-config.yaml
Outdated
There was a problem hiding this comment.
I don't mind if we do it in a follow-up PR, but I'd rather have something functional first and iterate on improvements later.
| rmm_version: | ||
| - "23.04.*" |
There was a problem hiding this comment.
Thanks John, done in 3707568 .
dependencies.yaml
Outdated
| specific: | ||
| - output_types: conda | ||
| matrices: | ||
| - matrix: | ||
| cuda: "11.8" | ||
| packages: | ||
| - cuda-sanitizer-api=11.8.86 | ||
| - matrix: | ||
| packages: |
dependencies.yaml
Outdated
| - checks | ||
| - cudatoolkit | ||
| - py_version | ||
| - run |
dependencies.yaml
Outdated
| - matrix: | ||
| packages: | ||
| - python>=3.8,<3.11 | ||
| run: |
dependencies.yaml
Outdated
| - python>=3.8,<3.11 | ||
| run: | ||
| common: | ||
| - output_types: [conda, requirements] |
ci/check_style.sh
Outdated
| conda activate checks | ||
|
|
||
| # Run pre-commit checks | ||
| pre-commit run --hook-stage manual --all-files --show-diff-on-failure |
No description provided.