Support interpolation methods of Accel and Gyro for IMU in SDK

[ruffsl]

Many conventional VIO pipelines intake internal measurements as synchronized pair of accelerometer and gyroscope streams. If the sample rates for these streams are unsynchronized, then a linear interpolation over time is often applied to unionize the two into a single measurement stream.

There's been a lot of duplicate work in downstream projects to facilitate this required abstraction:
IntelRealSense/realsense-ros#558
IntelRealSense/realsense-ros#1010

Currently, such abstractions must be implemented for each new downstream library:
intel/ros2_intel_realsense#103
#3921

As I'm at the cusp of porting this same filter from ROS1 to ROS2, I'd like to suggest we upstream this abstraction into the SDK so everyone can benefit from a complete and correct implementation for linear interpolation. Perhaps the sensor firmware itself could compute the filter, given the hardware always knows the SENSOR_TIMESTAMP for both a Accel and Gyro, circumventing failure modes from #5710 , where such filtering on the host side could be inaccurate from imprecise clock domains.

cc @doronhi

[RealSenseCustomerSupport]

---

Hi @ruffsl

Thanks for posting this request. For our T265 tracking solution, the current way in which we handle the IMU (Accel and Gyro) works well for our SLAM algorithm. The sample rates in which we've chosen is optimized for our SLAM algo and are fixed. No plans to make changes to our sample rates in firmware.

Looking at the information in the various github links you've provided, doronhi has provided great information regarding how the SDK is handling the IMU data as well.

[ruffsl]

Hello @RealSenseCustomerSupport ,

The sample rates in which we've chosen is optimized for our SLAM algo and are fixed. No plans to make changes to our sample rates in firmware.

This request does not pertain to changing existing sample rates for the Accel and Gyro, only that an additional frame type be made available that interpolates the two into a single 6DoF motion frame.

Looking at the information in the various github links you've provided, doronhi has provided great information regarding how the SDK is handling the IMU data as well.

I think this kind of filtering or preprocessing is a core necessity for many user end domain applications, no different than any of the existing filters like the temporal interpolation of depth frames. Here is a case example of where downstream has had to re-implement this same feature for a second time:

intel/ros2_intel_realsense#103

For our T265 tracking solution

At present, we happen to be using the D435i, so the firmware tracking for T265 is not applicable.

[RealSenseCustomerSupport]

---

Thanks for the update. Apologies for thinking this was pertaining to T265 product line. That was the context in which this thread was being viewed. We will look at it from the D435i standpoint.

[richard-elvira]

Hello @RealSenseCustomerSupport, I see that the SDK allows to interpolate the Inertial data to obtain sync measures from the gyroscope and accelerometer but I only find information in ROS and I need to obtain the synchronized IMU information without ROS.

I am working in a C++ project with RealSense T265. The code obtains the information by callbacks but the gyroscope and accelerometer are not synchronized. There is a way to apply the interpolate solution in that scenario.

Thanks.

[MartyG-RealSense]

Hi @ruffsl I am now handling this case and would like to provide you with an update.

There is an active Intel feature request relating to the requested feature of supporting interpolation methods of Accel and Gyro for the IMU in the RealSense SDK. There is no progress to report regarding this request at the time of writing this though.

[Nir-Az]

This feature was already add on ROS2 latest releases.
@MartyG-RealSense @ruffsl can you confirm this request can be closed?

[ruffsl]

This feature was already add on ROS2 latest releases.

I'm not currently using librealsense atm, but if you could link to the docs for this feature, then feel free to close.

However, is this what you are referring to:

https://github.com/IntelRealSense/realsense-ros/blob/047d601ede1a4b7735ee09bb053fa1850ae90ba6/README.md?plain=1#L383-L386

Seems a little orthogonal to this ticket, as this was again implemented external of the SDK, thus duplicating such filters.

https://github.com/IntelRealSense/realsense-ros/blob/047d601ede1a4b7735ee09bb053fa1850ae90ba6/realsense2_camera/src/base_realsense_node.cpp#L348

[Nir-Az]

Yes, the implementation is outside of the SDK and implemented in the ROS node as we saw most of the users asking for it wants it in the ROS wrapper.
We may rethink in the future if such capability should come from inside the SDK itself.
Thanks for the idea :)