Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure cudf internals use pylibcudf in pure Python mode #13909

Merged

Conversation

vyasr
Copy link
Contributor

@vyasr vyasr commented Aug 17, 2023

Description

In the short term, cudf internals will be the primary client of pylibcudf, and in the intermediate term cudf internals will likely remain the most exhaustive test of pylibcudf's API. While pylibcudf is designed to be used as either a Python or a Cython library, Python usage is expected to be the much more common entry point. Therefore, it would be best for cudf to use pylibcudf solely as a Python library. This is also useful because it would theoretically allow us to turn cudf into a pure Python library, or at least something much closer to it. In the long run we will likely want to leverage pylibcudf's Cython to accelerate parts of cudf, but that isn't something we should aim for the in the first pass. Switching to pure Python mode is also important because of certain limitations with Cython scoped enum support that will hopefully be fixed soon in Cython itself.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@vyasr vyasr added 3 - Ready for Review Ready for review by team Python Affects Python cuDF API. Cython improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Aug 17, 2023
@vyasr vyasr requested a review from a team as a code owner August 17, 2023 21:18
@vyasr vyasr self-assigned this Aug 17, 2023
@vyasr vyasr requested review from shwina and galipremsagar August 17, 2023 21:18
@vyasr vyasr force-pushed the feat/pylibcudf_dtype_conversion branch from f65e2f9 to c0816e9 Compare August 17, 2023 21:23
@shwina
Copy link
Contributor

shwina commented Aug 18, 2023

I support this proposal -- ultimately, I think we likely don't want two layers of Cython between libcudf and cuDF anyway.

@vyasr
Copy link
Contributor Author

vyasr commented Aug 21, 2023

/merge

@rapids-bot rapids-bot bot merged commit c2f2167 into rapidsai:branch-23.10 Aug 21, 2023
@vyasr vyasr deleted the feat/pylibcudf_dtype_conversion branch August 21, 2023 20:37
@vyasr vyasr added the pylibcudf Issues specific to the pylibcudf package label May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function non-breaking Non-breaking change pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants