CUDA support plan?

[SamuelLordBlack]

In another thread, a contributor wrote "we are working on adding out-of-the-box CUDA support and planning to publish it soon". How soon is soon, and which components are planned? Specifically, I'm trying to decide whether I should write my own implementation of align or wait for something official.

We're currently running 2.16.1.

Thanks.

[MartyG-RealSense]

After that mention of CUDA in June, CUDA optimized implementation was added for Jetson TX in version 2.13.0 on June 22 2018.

#1866

[SamuelLordBlack]

That's basically two functions - format conversion and point clouds. The remainder of the library has not been touched, and that's what I'm looking for.

[dorodnic]

Hi @SamuelLordBlack
Image conversions and projection functions are a logical place to start, as they are most commonly used performance hits. It makes sense to add alignment next, and it is taking longer than I originally expected. We are constantly receiving feedback from our users, and trying to prioritize things based on that. The project also welcomes and relies on user pull-requests to better support non-Intel platforms, so if you do write your own implementation please consider sharing it.
For the short term, our plans involve supporting more upcoming RealSense devices, continue investing in C# & Unity, expanding pypi distribution and Jupyter notebooks, build better integration with Open3D and OpenCV, improving CMake integration of samples and the Windows installer. In terms of optimizations my intuition was to invest in generic GPU acceleration next, but there is no official plan. Given more feedback from the Jetson community, we can go back to that instead.

[SamuelLordBlack]

Thanks, Sergey. I'll see what I can drum up from the Jetson community. The align call currently takes 200ms on the Jetson TX2, so it's quite a bottleneck.

[lesreaper]

Wow, that's a lot! I didn't think it was that big, though I did notice a slowdown on the TX2.

[dorodnic]

I agree this requires a fix. We are looking into the issue (I'm a bit hesitant to commit, because align flow is somewhat more complicated than what we done before, and I'm not CUDA expert by any means. But we will do our best)

[dorodnic]

CUDA optimized alignment is now part of librealsense 2.16.5.
@SamuelLordBlack thank you for bringing this to our attention.
We still don't have a strict CUDA support plan, but we are committed to addressing problems that are important to our customers.

[SamuelLordBlack]

Thanks! That's great turnaround time. We're looking forward to pulling that down.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi SamuelLordBlack,

If not yet, you may subscribe to RealSense SDK info and check back for CUDA related improvement.

If nothing else is needed, we will close this one soon.

Thank you!