Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Pin dask and distributed for release #1003

Merged
merged 8 commits into from
Oct 3, 2022
Merged

[REVIEW] Pin dask and distributed for release #1003

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
cd79266
pin dask and distributed for release
galipremsagar Sep 29, 2022
4f75696
fix
galipremsagar Sep 29, 2022
41ab17d
fix
galipremsagar Sep 29, 2022
c8588ad
fix
galipremsagar Sep 29, 2022
2a44872
fix
galipremsagar Sep 29, 2022
14cc910
Update ci/cpu/build.sh
galipremsagar Sep 29, 2022
be97091
Apply suggestions from code review
galipremsagar Sep 30, 2022
ac1b938
Apply suggestions from code review
galipremsagar Oct 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions ci/cpu/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ export CUDA_REL=${CUDA_VERSION%.*}
export GPUCI_CONDA_RETRY_MAX=1
export GPUCI_CONDA_RETRY_SLEEP=30

# Whether to keep `dask/label/dev` channel in the env. If INSTALL_DASK_MAIN=0,
# `dask/label/dev` channel is removed.
export INSTALL_DASK_MAIN=0

# Switch to project root; also root of repo checkout
cd "$WORKSPACE"

Expand All @@ -43,9 +47,13 @@ gpuci_logger "Activate conda env"
. /opt/conda/etc/profile.d/conda.sh
conda activate rapids

# Remove rapidsai-nightly channel if we are building main branch
# Remove `rapidsai-nightly` & `dask/label/dev` channel if we are building main branch
if [ "$SOURCE_BRANCH" = "main" ]; then
conda config --system --remove channels rapidsai-nightly
conda config --system --remove channels dask/label/dev
elif [[ "${INSTALL_DASK_MAIN}" == 0 ]]; then
# Remove `dask/label/dev` channel if INSTALL_DASK_MAIN=0
conda config --system --remove channels dask/label/dev
fi

gpuci_logger "Check compiler versions"
Expand All @@ -61,8 +69,8 @@ conda list --show-channel-urls
# FIX Added to deal with Anancoda SSL verification issues during conda builds
conda config --set ssl_verify False

pip install git+https://github.com/dask/dask.git@main
pip install git+https://github.com/dask/distributed.git@main
pip install git+https://github.com/dask/dask.git@2022.9.2
pip install git+https://github.com/dask/distributed.git@2022.9.2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe in a follow-up PR, but would it make more sense to use the same logic as we do in ci/gpu/build.sh, or is there a reason I'm not aware for keeping pip install?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think at this point these are redundant, I'm planning on getting these removed in 22.12 unpinning PRs across some repos which still do pip installs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, that would be even better. Thanks Prem!


################################################################################
# BUILD - Package builds
Expand Down
5 changes: 3 additions & 2 deletions ci/gpu/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,10 @@ export NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=1

# Install dask and distributed from main branch. Usually needed during
# development time and disabled before a new dask-cuda release.
export INSTALL_DASK_MAIN=1
export INSTALL_DASK_MAIN=0

# Dask version to install when `INSTALL_DASK_MAIN=0`
export DASK_STABLE_VERSION="2022.7.1"
export DASK_STABLE_VERSION="2022.9.2"

################################################################################
# SETUP - Check environment
Expand Down Expand Up @@ -77,6 +77,7 @@ if [[ "${INSTALL_DASK_MAIN}" == 1 ]]; then
else
gpuci_logger "gpuci_mamba_retry install conda-forge::dask==${DASK_STABLE_VERSION} conda-forge::distributed==${DASK_STABLE_VERSION} conda-forge::dask-core==${DASK_STABLE_VERSION} --force-reinstall"
gpuci_mamba_retry install conda-forge::dask==${DASK_STABLE_VERSION} conda-forge::distributed==${DASK_STABLE_VERSION} conda-forge::dask-core==${DASK_STABLE_VERSION} --force-reinstall
conda config --system --remove channels dask/label/dev
fi


Expand Down
4 changes: 2 additions & 2 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
dask>=2022.7.1
distributed>=2022.7.1
dask==2022.9.2
distributed==2022.9.2
pynvml>=11.0.0
numpy>=1.16.0
numba>=0.54