Amesos2: Improve handling of x and b copy or assign behavior

[MicheldeMessieres]

Enhancement

@trilinos/amesos2

@srajama1 @ndellingwood @kyungjoo-kim Opening this issue to keep track of one unresolved todo item for the Amesos2 refactoring to Kokkos views. When accessing x and b for the solve step, the new setup may obtain direct ptrs to the source multivector or copy the data, depending on whether memory space and types match.

The decision to copy or assign the ptr happens at a low level in the adapter. It would be useful for the solver to have some kind of API to know what is going to happen before actually making the request.

If x can be assigned directly (no copy) then we would just get x, but not need to put x after the solve, since we solved straight into the destination array. Currently we get x and put x. If x is copied, the get just needs an uninitialized space, not a copy of the multivector x which copies uninitialized data unnecessarily. If x is not copied (ptr assigned) then the put x is not necessary though it shouldn't be costing much since it's just a second ptr assignment.

Another example where this API would help is the upcoming superlu refactor. b may be modified in some cases by the solve. In those cases we must always use a copy of b, even if the memory space and types match. Otherwise the b input vector will be changed and the next solve cycle will fail. Currently always making a copy in this case but sometimes that will be redundant so the API would let us check and avoid the second copy if not necessary.

So improving this would be an increase in efficiency. I didn't work on making this setup yet because I was not sure how distributed data would fit into the scheme. It seemed better to wait until some of the other pieces were in place.

[MicheldeMessieres]

Resolved with #7678