Add cuda::mr::cuda_memory_resource

[miscco]

This introduces a new memory resource that allocates device memory through cudaMalloc / cudaFree

Fixes #1512

[miscco]

@harrism you might be interested in this

[harrism]

Nice addition. I didn't review the refactoring, only the new cuda_memory_resource. I think the refactoring is just splitting into multiple files, and that should be clarified in the PR description. It would have been nice to separate these into separate PRs.

Most of my suggestions are doc clarifications and fixes.

[harrism]

suggestion: I think allocation should throw, not assert. Also note that this assertion/error means that the alignment parameter is NOT ignored.

[harrism]

Question: Just because cudaMalloc aligns to 256 (or possibly 512, even though documented at 256), doesn't mean an MR can't support larger allocations. I suppose users who really want that can write a wrapping resource that aligns, but is there another reason we don't support it here? Seems a bit strange that cuda::mr has this parameter and our most basic implementation semi-ignores it and adds another method without it that is not part of the concept.

[miscco]

I added an exception / trap when we are facing an invalid alignment. I also merged the two allocate functions through a default argument

[jrhemstad]

Seems a bit strange that cuda::mr has this parameter and our most basic implementation semi-ignores it and adds another method without it that is not part of the concept.

The existence of the alignment parameter does not mean all possible alignment values are valid. Each resource implementation will have its own limitations and this includes alignment.

cuda_memory_resource is intended to be the simplest possible wrapper on top of cudaMalloc, and so it will inherit all the limitations of cudaMalloc, including alignment. So a user requesting an alignment up to 256 is fine, but beyond that, we throw.

We could add an additional resource that allows for larger alignments, (or an adapter!), but that would no longer be a trivial wrapper on top of cudaMalloc.

[jrhemstad]

suggestion: I think this check is unnecessary. It's illegal to ever call deallocate(p, N, M) for p that was not returned by p = allocate(N, M). Therefore, we would have already checked the alignment during the allocate call.

[miscco]

There is no guarantee that a user matches allocate and deallocate correctly.

This is just a tiny bit safer than without and it does not hurt

[harrism]

Suggestion: The semantics are pretty simple, can we just spell it out something like the following (or whatever the incantation would be)?

Suggested change

	* @return Comparison of both resources converted to a resource_ref<>
	* @return true if both arguments are convertible to resource_ref<>(cuda_memory_resource&), false otherwise

[miscco]

I went with: Result of equality comparison of both resources converted to a resource_ref<>

[harrism]

OK, but then the reader has to go dig up docs on the semantics of comparison of refs. Doesn't that just compare the referenced object types? Is there a section in the docs on equality comparison that you could link to?

std::pmr::memory_resource and RMM (current) memory resources have very different comparison semantics, so I think this should be clearly documented.

[harrism]

Typo in issue title (hyphen should be underscore).