-
Notifications
You must be signed in to change notification settings - Fork 540
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cuml: Build CUDA 12 packages #5318
cuml: Build CUDA 12 packages #5318
Conversation
This PR needs to be updated to pull artifacts from rapidsai/raft#1388 once those are ready, and it needs a cumlprims PR as well. |
ba8f9b0
to
0a44511
Compare
There was one pytest failure that seems relevant and we need to investigate it but it's not a build blocker to get packages ready. Opened issue #5497 and skipped the pytest temporarily so we can get CUDA 12 packages early for testing. |
Follow up comment about the prior failed pytest, it happened to be an OOM orthogonal to CUDA 12. @viclafargue submitted a fix in #5498 and I merged the fix into this PR as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome progress. Comments attached.
Co-authored-by: Bradley Dice <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is getting super close!
conda/recipes/cuml/meta.yaml
Outdated
{% if cuda_major == "11" %} | ||
- cuda-python ==11.7.1 | ||
- cudatoolkit | ||
- libcublas {{ cuda11_libcublas_host_version }} | ||
- libcublas-dev {{ cuda11_libcublas_host_version }} | ||
- libcurand {{ cuda11_libcurand_host_version }} | ||
- libcurand-dev {{ cuda11_libcurand_host_version }} | ||
- libcusolver {{ cuda11_libcusolver_host_version }} | ||
- libcusolver-dev {{ cuda11_libcusolver_host_version }} | ||
- libcusparse {{ cuda11_libcusparse_host_version }} | ||
- libcusparse-dev {{ cuda11_libcusparse_host_version }} | ||
{% else %} | ||
- cuda-python ==12.0.0 | ||
- libcublas-dev | ||
- libcurand-dev | ||
- libcusolver-dev | ||
- libcusparse-dev | ||
{% endif %} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we actually need the -dev
packages for cuml? Shouldn't we only require the runtime packages? The dependency on -dev
packages should be isolated to libcuml I believe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I vaguely recall that the -dev
packages are needed from when I looked at this months ago. I think cuml might do something with raft that requires these libraries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't remember well, so did a commit trying it to check
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like we were able to remove this?
Co-authored-by: Vyas Ramasubramani <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One more round of questions about math library dependencies.
conda/recipes/cuml/meta.yaml
Outdated
- libcublas {{ cuda11_libcublas_host_version }} | ||
- libcurand {{ cuda11_libcurand_host_version }} | ||
- libcusolver {{ cuda11_libcusolver_host_version }} | ||
- libcusparse {{ cuda11_libcusparse_host_version }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do any of these CUDA math libraries need to be in host
for cuml Python? Are they directly used by cuml Python or only indirectly through libcuml? If the latter, maybe we can get away with only specifying them in libcuml's recipe.
I wonder if these (non-dev) libraries are also remnants of the changes used when migrating to GitHub Actions before RAFT recipes were complete. It would surprise me if these are used directly in cuml Python, since they're not mentioned in python/CMakeLists.txt
. However, I see that there are linkages in the installed packages: for example, ldd manifold/umap.cpytho-310-x86_64-linux-gnu.so
shows libcufft.so.10
, libcurand.so.10
, libcusolver.so.11
, libcublas.so.11
, libcusparse.so.11
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After thinking about this more, we probably want to remove these. Putting “runtime” (non-dev) libraries in the host
section probably does nothing, except perhaps satisfy conda linking checks (which we can verify by looking at the warnings).
conda/recipes/cuml/meta.yaml
Outdated
{% if cuda_major == "11" %} | ||
- cuda-python ==11.7.1 | ||
- cudatoolkit | ||
- libcublas {{ cuda11_libcublas_host_version }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is libcufft missing? If so we'd need to add it to conda_build_config.yaml
for cuml, too. (See also my other comment about whether any of these libraries are needed in the Python packaging.)
- libcublas {{ cuda11_libcublas_host_version }} | |
- libcublas {{ cuda11_libcublas_host_version }} | |
- libcufft {{ cuda11_libcufft_host_version }} |
conda/recipes/cuml/meta.yaml
Outdated
- libcusparse {{ cuda11_libcusparse_host_version }} | ||
{% else %} | ||
- cuda-python ==12.0.0 | ||
- libcublas |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is libcufft missing? (Or can all these math libraries be removed from the Python packaging, see other comment?)
- libcublas | |
- libcublas | |
- libcufft |
Co-authored-by: Bradley Dice <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the hard work on this @dantegd! LGTM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything looks good for me too (can't approve since it's my PR).
/merge |
Thanks everyone! 🙏 |
This PR builds
libcuml
andcuml
conda packages using CUDA 12. Resolves #5234.