Add __cuda_array_interface__ to Buffer

[jakirkham]

Adds the __cuda_array_interface__ to Buffer. This simplifies its coercion to Numba arrays, which is then leveraged to get away from some legacy code in RMM around array handling.

[jakirkham]

We seem to be hitting some sort of edge case here. Tried switching to the utility function used elsewhere, but it seems to have issues comparing. Have a rough idea of the issue, but additional pointers would be welcome. 🙂

______________ test_reflected_ops_scalar[<lambda>-int160-Series] _______________

func = <function <lambda> at 0x7f531193f320>, dtype = <class 'numpy.int16'>
obj_class = 'Series'

    @pytest.mark.parametrize("obj_class", ["Series", "Index"])
    @pytest.mark.parametrize("func, dtype", list(product(_reflected_ops, _dtypes)))
    def test_reflected_ops_scalar(func, dtype, obj_class):
        # create random series
        np.random.seed(12)
        random_series = utils.gen_rand(dtype, 100, low=10)
    
        # gpu series
        gs = Series(random_series)
    
        # class typing
        if obj_class == "Index":
            gs = as_index(gs)
    
        gs_result = func(gs)
    
        # class typing
        if obj_class == "Index":
            gs = Series(gs)
    
        # pandas
        ps_result = func(random_series)
    
        # verify
>       utils.assert_eq(ps_result, gs_result)

cudf/tests/test_binops.py:380: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = 0     True
1     True
2     True
3     True
4     True
      ... 
95    True
96    True
97    True
98    True
99    True
Length: 100, dtype: bool

    def __nonzero__(self):
        raise ValueError(
            "The truth value of a {0} is ambiguous. "
            "Use a.empty, a.bool(), a.item(), a.any() or a.all().".format(
>               self.__class__.__name__
            )
        )
E       ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

/conda/envs/gdf/lib/python3.7/site-packages/pandas/core/generic.py:1555: ValueError

[kkraus14]

I'm guessing we're somehow getting down to this condition: https://github.com/rapidsai/cudf/blob/branch-0.13/python/cudf/cudf/tests/utils.py#L77

What are the types of ps_result and gs_result?

[jakirkham]

Sorry for the slow reply.

Actually it is getting stuck here. The comparison works, but the result is a Pandas Series, which can't be converted to a bool.

Backing up a bit, gs_result is a cuDF Series. However ps_result appears to be a NumPy array. Is the latter expected?

FWIW it appears this has always been the case. Not sure what caused us to now stumble into that.

[jakirkham]

Should add forcing ps_result to a Pandas Series fixes the issue.

[kkraus14]

Do you know what op this is that's causing the issue? It's a bit surprising that your PR changes are causing this to change behavior.

[jakirkham]

I tried locally with just lambda x: 1 + x, but would guess any of them have this issue.

Are we sure this is suppose to be gen_rand and not gen_rand_series though? ( #4091 )

[kkraus14]

🤷‍♂ not sure but if gen_rand_series fixes it that sounds good to me. This test is already pretty janky to test both indexes and series.

[jakirkham]

Yeah me neither honestly. So I could easily be wrong 🙂

Not that we should pick up more things here, but would it makes sense to break this test into two separate ones to handle Series and Index? If so, happy to raise an issue.

[kkraus14]

Possibly in the future, but the level of work for the reward is low right now. All of our index binaryops are implemented by casting to/from Series so once we fix that we can possibly address this.

stale

[jakirkham]

Given the __cuda_array_interface__ bits are in and this now is mostly a PR about using CuPy, have extracted those bits into a fresh draft PR ( #4110 ). We can then consider how we want to proceed with that on its own merits.