Remove versioneer #1204

pentschev · 2023-06-26T11:11:24Z

wence-

Thanks Peter.

raydouglass · 2023-06-26T14:08:17Z

pyproject.toml

@@ -3,7 +3,6 @@ build-backend = "setuptools.build_meta"
 requires = [
    "setuptools>=64.0.0",
    "tomli  ; python_version < '3.11'",
-    "versioneer>=0.24",
 ]

 [project]


Since version was removed from setup.py, I believe it needs to be added here instead (eg https://github.com/rapidsai/cudf/blob/branch-23.08/python/dask_cudf/pyproject.toml#L12). And I think the dynamic field should be removed.

Also, an entry needs to be added to update-version.sh to update this file's version too.

You can see that no version is being used in the wheel build: https://github.com/rapidsai/dask-cuda/actions/runs/5377443612/jobs/9755962220?pr=1204#step:4:54

Also, we need to ensure that nightlies have the a${RAPIDS_DATE_STRING} appended.

Thanks @raydouglass for catching that. I've done the changes in 210937e and it looks like the build version is correct now: https://github.com/rapidsai/dask-cuda/actions/runs/5379525987/jobs/9760908314#step:4:54 .

We have RAPIDS_DATE_STRING being passed to the package version in meta.yaml, which seems consistent with cuDF. Is there anywhere else we need to pass it?

Yes, since we upload nightlies to pypi.org, they need a "pre-release" version. This would be something like 23.8.0a230626 (here).

I'm not exactly sure the best way to implement this. We bypass this issue in other rapids libraries by uploading them to our own nightly index and they use a unix epoch in the version to distinguish, but they are not pre-releases.

I don't know if this is acceptable, but I've attempted something in 937919b . That change will essentially change version in pyproject.toml before building the wheel and later revert it. I don't know how we would identify whether we are doing a release build (and thus not modify pyproject.toml) or a pre-release build that we want to do that change, any ideas?

Obviously, we would also need to update ci/release/update-version.sh to do the necessary changes in this file if we go ahead with that solution, but want to check first whether this is how we want to move ahead.

cc @vyasr who may have insights here.

Regarding how you can decide whether or not to make the change, inside this script you have access to a bash function that you can use: if rapids-is-release-build; do ...; done.

I think the current solution is acceptable. In the long run I would like to move the rest of RAPIDS to proper pre-releases, but that will require additional changes to our dependency management so I hadn't considered the practical question of how we would do it. The workflows for other RAPIDS packages are just different enough that the exact approach would probably end up being slightly different.

Nice, you guys thought of everything! 😄

…ioneer

pentschev · 2023-06-29T20:27:45Z

I think everything is correct now, including wheel version.

Would you mind taking a look @vyasr @raydouglass ?

raydouglass · 2023-06-29T20:33:49Z

ci/build_python_pypi.sh

@@ -8,11 +8,36 @@ python -m pip install build --user
 export GIT_DESCRIBE_TAG=$(git describe --abbrev=0 --tags)
 export GIT_DESCRIBE_NUMBER=$(git rev-list ${GIT_DESCRIBE_TAG}..HEAD --count)

+# Build date for PyPI pre-releases
+export RAPIDS_VERSION_NUMBER="23.08"
+export RAPIDS_FULL_VERSION_NUMBER="${RAPIDS_VERSION_NUMBER}.00"


Can't hardcode the patch version (unless you also update via update-version.sh but we do not do this in any other repo).

Since during a release, update-version.sh is run before building, you could pull the current version from pyproject.toml. Then only adjust it to the pre-release version for not release builds.

The version is being updated in https://github.com/rapidsai/dask-cuda/pull/1204/files#diff-8a0188e09c831b0e6ca8c1c51ab640efd1e519b1d7f94ab6251c9f0b90a82304R51 . I have no preference for getting the version from pyproject.toml or updating it with update-version.sh (except the latter is already done, and perhaps slightly easier). Do you have a strong preference for the pyproject.toml option?

I have a weak pref for pyproject.toml, not a strong one. FWIW here's how we do it for our other wheels: https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-pip-wheel-version

vyasr

Couple of small things.

vyasr · 2023-06-30T00:40:19Z

ci/build_python_pypi.sh

@@ -8,11 +8,36 @@ python -m pip install build --user
 export GIT_DESCRIBE_TAG=$(git describe --abbrev=0 --tags)
 export GIT_DESCRIBE_NUMBER=$(git rev-list ${GIT_DESCRIBE_TAG}..HEAD --count)

+# Build date for PyPI pre-releases
+export RAPIDS_VERSION_NUMBER="23.08"
+export RAPIDS_FULL_VERSION_NUMBER="${RAPIDS_VERSION_NUMBER}.00"


I have a weak pref for pyproject.toml, not a strong one. FWIW here's how we do it for our other wheels: https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-pip-wheel-version

vyasr · 2023-06-30T00:41:09Z

ci/build_python_pypi.sh

+# Inplace sed replace; workaround for Linux and Mac
+function sed_runner() {
+    sed -i.bak ''"$1"'' $2 && rm -f ${2}.bak
+}


Let's avoid proliferating the sed_runner function. It has problems, and we don't need it here. This script is only ever run in CI on Linux machines. Let's just use plain sed for now.

Done in d4d887a .

pentschev · 2023-06-30T12:29:49Z

I have a weak pref for pyproject.toml, not a strong one. FWIW here's how we do it for our other wheels: https://github.com/rapidsai/gha-tools/blob/main/tools/rapids-pip-wheel-version

Done in 4117cf4 .

@raydouglass @vyasr would you mind taking another look?

pentschev · 2023-06-30T14:58:08Z

Thanks @raydouglass @vyasr @wence- !

pentschev · 2023-06-30T14:58:14Z

/merge

Proposes removing this project's build-time dependency on `tomli`. It appears to no longer be necessary. ```shell git grep tomli ``` ## Notes for Reviewers I originally noticed something similar in `ucx-py` (rapidsai/ucx-py#1042), then went searching for similar cases across RAPIDS. I'm not sure why this project has a dependency on `tomli`, but I suspect it was related to the use of `versioneer` in this project's history. Reference: python-versioneer/python-versioneer#338 (comment) This project doesn't use `versioneer` any more (#1204). I strongly suspect that the dependency on `tomli` can be removed. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - https://github.com/jakirkham - Ray Douglass (https://github.com/raydouglass) URL: #1338

pentschev added 2 commits June 26, 2023 03:25

Remove versioneer

88f4ce4

Update release script

b7499d6

pentschev requested review from a team as code owners June 26, 2023 11:11

github-actions bot added python python code needed conda conda issue ci labels Jun 26, 2023

pentschev added 3 - Ready for Review Ready for review by team feature request New feature or request non-breaking Non-breaking change labels Jun 26, 2023

wence- approved these changes Jun 26, 2023

View reviewed changes

raydouglass requested changes Jun 26, 2023

View reviewed changes

pentschev added 7 commits June 26, 2023 07:37

Add version to pyproject.toml

210937e

Modify build date during PyPI builds for pre-release packages

937919b

Handle both release and nightly PyPI versions

662d212

Merge remote-tracking branch 'upstream/branch-23.08' into remove-vers…

7c542ee

…ioneer

Fix variable names

f5e67ba

Fix version conditions

c43d21b

Fix path

f4f61c0

raydouglass reviewed Jun 29, 2023

View reviewed changes

vyasr requested changes Jun 30, 2023

View reviewed changes

pentschev added 2 commits June 30, 2023 05:12

Remove sed_runner form PyPI build script

d4d887a

Use pyproject.toml's version as source for PyPI pre-releases

4117cf4

vyasr approved these changes Jun 30, 2023

View reviewed changes

raydouglass approved these changes Jun 30, 2023

View reviewed changes

rapids-bot bot merged commit e13f500 into rapidsai:branch-23.08 Jun 30, 2023

pentschev deleted the remove-versioneer branch June 30, 2023 21:15

jameslamb mentioned this pull request May 6, 2024

remove 'tomli' dependency #1338

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove versioneer #1204

Remove versioneer #1204

pentschev commented Jun 26, 2023

wence- left a comment

raydouglass Jun 26, 2023

raydouglass Jun 26, 2023

pentschev Jun 26, 2023

raydouglass Jun 26, 2023

pentschev Jun 26, 2023

vyasr Jun 29, 2023 •

edited

Loading

pentschev Jun 29, 2023

pentschev commented Jun 29, 2023

raydouglass Jun 29, 2023

pentschev Jun 29, 2023

vyasr Jun 30, 2023

vyasr left a comment

vyasr Jun 30, 2023

vyasr Jun 30, 2023

pentschev Jun 30, 2023

pentschev commented Jun 30, 2023

pentschev commented Jun 30, 2023

pentschev commented Jun 30, 2023

Remove versioneer #1204

Remove versioneer #1204

Conversation

pentschev commented Jun 26, 2023

wence- left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vyasr Jun 29, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pentschev commented Jun 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vyasr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pentschev commented Jun 30, 2023

pentschev commented Jun 30, 2023

pentschev commented Jun 30, 2023

vyasr Jun 29, 2023 •

edited

Loading