X500 corrupted RGBD data

[m3d]

Hello,
we are observing corrupted RGBD data of up/down cameras in CTU_CRAS_NORLAB_X500_SENSOR_CONFIG_1 model. They are broken even after set_rate reduction. Example of sequence of depth data in Tunnel Simple 2 looks like this:

This corresponds to saving of /ROBOTNAME/up_rgbd/optical/depth/image_raw. Thanks in advance for any hint

[peci1]

was the robot rotating in yaw?

[m3d]

was the robot rotating in yaw?

yes, but not much - should it matter?

[peci1]

Yes, it does - there is a known issue where orientation w.r.t world frame affects camera clipping: gazebosim/gz-sensors#128 .

[peci1]

A workaround is to set very high near clip distance, e.g. 0.5 m was usually enough, but the you lose depth measurements if you get closer than that...

[m3d]

Interesting, thanks for hint - I would try it tonight.

[m3d]

A workaround is to set very high near clip distance, e.g. 0.5 m was usually enough, but the you lose depth measurements if you get closer than that...

Another stupid question - is it possible to do it on simulation side? Or do you mean to drop whole image or part of the image "once a while"? Is there some service like the newest set_rate??

[peci1]

This would have to be set in the SDF. If running locally, you can edit the SDF in ~/.ignition/fuel .

[m3d]

OK, thanks - so no chance on CloudSim?

[peci1]

I think this is a regular bug so you can try submitting a fix for it.

[peci1]

Do you know what the clipped values are? Are they NaN, Inf or something else?

[m3d]

Do you know what the clipped values are? Are they NaN, Inf or something else?

well, I have only indirect logs, as the depth is converted to 16bit millimeters and

>>> depth
array([[1, 1, 1, ..., 1, 1, 1],
       [1, 1, 1, ..., 1, 1, 1],
       [1, 1, 1, ..., 1, 1, 1],
       ...,
       [1, 1, 1, ..., 1, 1, 1],
       [1, 1, 1, ..., 1, 1, 1],
       [1, 1, 1, ..., 1, 1, 1]], dtype=uint16)

which is after

>>> np.clip(float('inf'), 1, 0xFFFF)
65535.0
>>> np.clip(float('-inf'), 1, 0xFFFF)
1.0
>>> np.clip(float('nan'), 1, 0xFFFF)
nan

and then conversion to uint16

>>> arr = np.array([float('nan')])
>>> np.ndarray.astype(arr, dtype=np.dtype('H'))
array([0], dtype=uint16)

So I suppose that it was -inf for up pointing camera.

[m3d]

@nkoenig is there a plan to fix this before the first SubT code freeze? Thanks m.

[knoedler]

#960 will fix a lot of what you are seeing with the top camera. The camera is occluded by the visual.

[m3d]

@knoedler thanks for hope :-)

[angelacmaio]

#960 was merged

[m3d]

it was merged but not released?! (which may change today or tomorrow)

[malcolmst]

any information on when this will be released?

[malcolmst]

@angelacmaio @nkoenig sorry to be a pain about this, but whether or not #960 will be released before the upcoming submission deadline will affect my work planning significantly, so if you have any information on this it would be greatly appreciated. Thanks!

[m3d]

This issue seems to be really gone with release 2021-07-13, thank you.