[FEA] CUDF API to get allocation size needed for copy to host. #8885
Labels
reliability
Features to improve reliability or bugs that severly impact the reliability of the plugin
task
Work required that improves the product but is not user facing
Is your feature request related to a problem? Please describe.
There are a number of places where we try to copy a GPU column vector to the host to do something with it. This is likely to involve allocating multiple buffers. And technically if we do multiple blocking allocations we need to make the other outstanding allocations spillable before we call the blockingAlloc API, or else we could get a deadlock. That is just too complicated to try and capture in the CUDF java API code. Instead we should do a reservation on the amount of data needed and then allocate it from that reservation. So we need an API that is fast and would tell us exactly how much memory we would need to reserve. Then we could do the reservation and when we copy it to the host we could replace the allocation API #8884 with one that uses the reservation. We could also try to do something like the chunked pack code/contig split code instead, but I think this is the fastest path to get something working.
The text was updated successfully, but these errors were encountered: