-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] handling CUDA-suffixed dependencies needed at setuptools.setup()
time
#39
Comments
Something seems off here. Let me try and reproduce locally. The sequence here seems to suggest that somehow it's going down a legacy code path. |
I'll post my (edited) Slack comment here for the record. What we have here is the original chicken and egg problem of That being said, it seems that the setuptools PEP 517 build backend ( The only really viable solution I see is to special-case RBB's |
Adding some links for my own reference. Here's where That calls which ends up reading in the |
Description
Consider a project with the following characteristics:
setup.py
runningsetuptools.setup()
import {library}
in thatsetup.py
, prior tosetup()
being called{library}
is only distributed via CUDA-suffixed wheels (e.g.{library}-cu11
)How should such cases be handled? Is it possible to make them work with
rapids-build-backend
?Details
Consider the specific case of
ucx-py
.Its
setup.py
includes stuff like this that runs beforesetuptools.setup()
is executed:(ucx-wheels/setup.py)
The key line is that
import libucx
.By the time that line runs,
libucx-cu11
orlibucx-cu12
needs to have been installed. It that seemspip wheel
ends up runningsetup.py
beforerapids-build-backend
has had a chance to replace the contents ofpyproject.toml
.With
libucx==1.15.0
(unsuffixed) in[build-system]
, like this:Wheel-building fails like this:
(example build link)
With it omitted from that table and moved down into
[tool.rapids-build-backend]
, wheel building fails like this:(example build link)
Options I can think of
(in no particular order)
rapids-build-backend
, but preserve thesed
-replacing of justlibucx
in wheel-building scriptsrapids-dependency-file-generator --matrix "cuda=${RAPIDS_CUDA_VERSION%.*}"
in build scripts prior topip wheel
to update
pyproject.toml
in placelibucx
at build time by some other mechanism, omit it from[build-system]
table, runpip wheel --no-build-isolation
setup.py
, customize the command run bypython setup.py build_ext
import libucx
stuff when it's called to build extensions (instead of whensetuptools.setup()
is called)Request for comment
Other options?
Any of these that definitely won't work?
How should we proceed?
References
Created after starting rapidsai/ucx-py#1044.
The text was updated successfully, but these errors were encountered: