[FEA] [Java] Add a way to allocate via cudaMalloc for device memory buffers

[abellina]

Given #9201, we are starting to experiment with the CUDA Async Allocator. cudaMallocAsync does not support GPUDirect RDMA, so we would like to be able to allocate some memory (i.e. bounce buffers) directly using cudaMalloc.

One approach to get this done is to expose a DeviceMemoryBuffer.cudaAllocate or DeviceMemoryBuffer.cudaMalloc, that implies bypassing RMM, in order to retain the GPUDirect RDMA functionality in certain cases (RAPIDS Spark with UCX)

[jrhemstad]

Why bypass RMM? You can use more than resource at a time. So just use cuda_memory_resource to allocate your bounce buffer and cuda_async_resource for your other allocations.

[abellina]

Why bypass RMM? You can use more than resource at a time. So just use cuda_memory_resource to allocate your bounce buffer and cuda_async_resource for your other allocations.

Everything we have right now assumes the RMM pool is a singleton, and on top of that, that there's a single resource. Because of that it seemed simpler to limit the RMM pool to the amount of free memory - the amount needed for GPUDirect RDMA, and allocating directly.

That said, we do want to make Rmm.java more generic (#9209). It would mean more work, but it may make it easier for others to consume, and would fit more or less with what you are saying. The downstream app (spark-rapids in this case), could compose the allocators it needs under Rmm.

[abellina]

Note the way this was implemented was via the cuda_memory_resource in RMM.