[FEA] Add Cython / Python wrappers for rmm::cuda_stream_pool

[harrism]

I refrained from doing the Python for the CUDA stream pool added in #659 myself because I'm not sure what the right approach is for Python. But I think we should add a Python interface, so I'm opening this issue for discussion.

CC the usual suspects: @shwina @kkraus14 @pentschev @jakirkham

[harrism]

Also CC @teju85 @afender @cjnolet as potential users.

[jakirkham]

Thanks! Yeah checking with Peter when he's back from holiday sounds like the right approach 🙂

cc @quasiben (for vis)

[pentschev]

I think this is a very useful feature for Python too. I think a good initial approach is to treat it as we do for the memory pool, at rmm.reinitialize the user decides whether to use a stream pool or not, then asking for a new non-default Stream object will return a stream from the pool. I think I may be oversimplifying things in my head and overlooking potential pitfalls, but nevertheless I think we may want to try to simplify usability in Python.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[kkraus14]

@shwina is this done with the new Cython classes for owning / non-owning streams?

[shwina]

@isVoid and I are still investigating stream use in Python, and working on a stream pool abstraction.

In our initial work, we found it more convenient to just use a Python list of Stream objects to hold a stream pool. This is because a Stream object in Python can be backed by more than just an rmm::cuda_stream, including Numba or CuPy owned streams. Whereas rmm::cuda_stream_pool is strictly a wrapper around a vector of rmm::cuda_stream objects.

We can keep this issue open until we have an implementation finalized, but I think we can close it safely.

[kkraus14]

In our initial work, we found it more convenient to just use a Python list of Stream objects to hold a stream pool. This is because a Stream object in Python can be backed by more than just an rmm::cuda_stream, including Numba or CuPy owned streams. Whereas rmm::cuda_stream_pool is strictly a wrapper around a vector of rmm::cuda_stream objects.

Yes, but if everyone is creating their own pool of streams, that's quite a bit of streams to create which is non-free. Ideally we could all share a reasonably sized pool of streams in the future 😄

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[jakirkham]

PR ( #818 ) appears relevant

cc @divyegala (for awareness)

[divyegala]

@jakirkham my use-case only needs the cython extensions for rmm::cuda_stream_pool. Should I also write the python extended class for it?

[jakirkham]

Starting with Cython seems fine. Can always be extended for Python later as needed

[harrism]

Should we close this with #818, or do we need more Python?

[shwina]

There's no (immediate) need for a Python abstraction around a cuda_stream_pool in cuDF, but perhaps other libraries need it? @divyegala?

[divyegala]

@shwina there's no immediate need in RAFT/cuML/cuGraph either

[shwina]

Thanks! Closing this out then.