Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] CUDF API to get allocation size needed for copy to host. #8885

Closed
revans2 opened this issue Jul 31, 2023 · 1 comment
Closed

[FEA] CUDF API to get allocation size needed for copy to host. #8885

revans2 opened this issue Jul 31, 2023 · 1 comment
Assignees
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin task Work required that improves the product but is not user facing

Comments

@revans2
Copy link
Collaborator

revans2 commented Jul 31, 2023

Is your feature request related to a problem? Please describe.
There are a number of places where we try to copy a GPU column vector to the host to do something with it. This is likely to involve allocating multiple buffers. And technically if we do multiple blocking allocations we need to make the other outstanding allocations spillable before we call the blockingAlloc API, or else we could get a deadlock. That is just too complicated to try and capture in the CUDF java API code. Instead we should do a reservation on the amount of data needed and then allocate it from that reservation. So we need an API that is fast and would tell us exactly how much memory we would need to reserve. Then we could do the reservation and when we copy it to the host we could replace the allocation API #8884 with one that uses the reservation. We could also try to do something like the chunked pack code/contig split code instead, but I think this is the fastest path to get something working.

@revans2 revans2 added feature request New feature or request task Work required that improves the product but is not user facing reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels Jul 31, 2023
@revans2
Copy link
Collaborator Author

revans2 commented Aug 23, 2023

This was fixed by rapidsai/cudf#13919

@revans2 revans2 closed this as completed Aug 23, 2023
@sameerz sameerz removed the feature request New feature or request label Oct 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
reliability Features to improve reliability or bugs that severly impact the reliability of the plugin task Work required that improves the product but is not user facing
Projects
None yet
Development

No branches or pull requests

3 participants