Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

I can't see any images or point clouds from a D455 connected to wi-fi #2357

Closed
Alpaca-kun opened this issue May 23, 2022 · 12 comments
Closed

I can't see any images or point clouds from a D455 connected to wi-fi #2357

Alpaca-kun opened this issue May 23, 2022 · 12 comments
Labels
D400

Comments

@Alpaca-kun
Copy link

Hi everybody,
I am a beginner in this area, so if you find any mistakes in my arguments, please correct me.

Currently, I am working with a TurtleBot3 and a Realsense D455.
I want to get data from the D455, connected into the TurtleBot3, and send it to my remote computer to build 3D maps.

To do this, I am accessing the TurtleBot3 over wi-fi from a remote computer in the same network.
From the remote computer I can see the ros2 topics published by the D455 and select them on the Rviz2.

However, I can’t see any image or point cloud on the Rviz2.
Any suggestions or documentations that can I follow?

Below is the information from my computers and the outputs from some commands tested.

Hardware specifications

TurtleBot3 (Raspberry pi 3B+)

  • Ubuntu Server 20.04
  • ROS2 Foxy Base (without GUI tools)
  • Realsense SDK 2.5.0 (installed from source)
  • ROS2 Wrapper for Intel® RealSense™ Devices (installed ros2-beta version from source)

Remote computer

  • Ubuntu 20.04
  • ROS2 Foxy Desktop
  • Realsense SDK 2.5.0 (installed from source)
  • ROS2 Wrapper for Intel® RealSense™ Devices (installed ros2-beta version from source)

Realsense camera D455

  • Connected to the Raspberry pi 3B+ via a USB 2.0 port
  • Usually, the below command is used to start the camera node:
ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true

Outputs from some commands

  • ros2 topic echo /camera/depth/color/points
    Nothing is shown, the echo command keeps a blank screen.

  • ros2 topic info -v /camera/depth/color/points

    Click here to expand 
    Type: sensor_msgs/msg/PointCloud2

Publisher count: 1

Node name: camera
Node namespace: /camera
Topic type: sensor_msgs/msg/PointCloud2
Endpoint type: PUBLISHER
GID: 01.0f.26.08.4e.5b.32.4e.01.00.00.00.00.00.13.03.00.00.00.00.00.00.00.00
QoS profile:
Reliability: RMW_QOS_POLICY_RELIABILITY_RELIABLE
Durability: RMW_QOS_POLICY_DURABILITY_VOLATILE
Lifespan: 2147483651294967295 nanoseconds
Deadline: 2147483651294967295 nanoseconds
Liveliness: RMW_QOS_POLICY_LIVELINESS_AUTOMATIC
Liveliness lease duration: 2147483651294967295 nanoseconds

Subscription count: 1

Node name: rviz
Node namespace: /
Topic type: sensor_msgs/msg/PointCloud2
Endpoint type: SUBSCRIPTION
GID: 01.0f.b5.be.f2.aa.c8.62.01.00.00.00.00.00.1f.04.00.00.00.00.00.00.00.00
QoS profile:
Reliability: RMW_QOS_POLICY_RELIABILITY_RELIABLE
Durability: RMW_QOS_POLICY_DURABILITY_VOLATILE
Lifespan: 2147483651294967295 nanoseconds
Deadline: 2147483651294967295 nanoseconds
Liveliness: RMW_QOS_POLICY_LIVELINESS_AUTOMATIC
Liveliness lease duration: 2147483651294967295 nanoseconds

  • rqt_graph of D455 and rviz2

    Click here to see image

    d455_rviz2
@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented May 23, 2022 •
edited
Loading

Hi @Alpaca-kun Can you post a copy of your ros2 launch log from the ROS terminal into a comment below, please? Thanks!

@Alpaca-kun
Copy link
Author

@MartyG-RealSense Thank you for your reply!
Here is the log of ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true

ros2 launch log 
ubuntu@ubuntu:~$ ros2 launch realsense2_camera rs_launch.py pointcloud.enable:=true
[INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2022-05-24-00-08-42-281537-ubuntu-13709
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [13711]
[realsense2_camera_node-1] [INFO] [1653350923.480822909] [camera.camera]: RealSense ROS v4.0.4
[realsense2_camera_node-1] [INFO] [1653350923.481280788] [camera.camera]: Built with LibRealSense v2.50.0
[realsense2_camera_node-1] [INFO] [1653350923.481410272] [camera.camera]: Running with LibRealSense v2.50.0
[realsense2_camera_node-1] [INFO] [1653350923.770922756] [camera.camera]: Device with serial number 141322252836 was found.
[realsense2_camera_node-1]
[realsense2_camera_node-1] [INFO] [1653350923.771147139] [camera.camera]: Device with physical ID 1-1.1.3-5 was found.
[realsense2_camera_node-1] [INFO] [1653350923.771231464] [camera.camera]: Device with name Intel RealSense D455 was found.
[realsense2_camera_node-1] [INFO] [1653350923.773645814] [camera.camera]: Device with port number 1-1.1.3 was found.
[realsense2_camera_node-1] [INFO] [1653350923.773890613] [camera.camera]: Device USB type: 2.1
[realsense2_camera_node-1] [WARN] [1653350923.774042285] [camera.camera]: Device 141322252836 is connected using a 2.1 port. Reduced performance is expected.
[realsense2_camera_node-1] [INFO] [1653350923.783332170] [camera.camera]: getParameters...
[realsense2_camera_node-1] [INFO] [1653350923.785419686] [camera.camera]: JSON file is not provided
[realsense2_camera_node-1] [INFO] [1653350923.785710841] [camera.camera]: Device Name: Intel RealSense D455
[realsense2_camera_node-1] [INFO] [1653350923.785873555] [camera.camera]: Device Serial No: 141322252836
[realsense2_camera_node-1] [INFO] [1653350923.785990225] [camera.camera]: Device physical port: 1-1.1.3-5
[realsense2_camera_node-1] [INFO] [1653350923.786081999] [camera.camera]: Device FW version: 05.13.00.50
[realsense2_camera_node-1] [INFO] [1653350923.786158564] [camera.camera]: Device Product ID: 0x0B5C
[realsense2_camera_node-1] [INFO] [1653350923.786232994] [camera.camera]: Sync Mode: Off
[realsense2_camera_node-1] [INFO] [1653350926.457541656] [camera.camera]: Set Exposure.1 to 7500
[realsense2_camera_node-1] [INFO] [1653350926.639088287] [camera.camera]: Set ROS param depth_module.profile to default: 640x480x15
[realsense2_camera_node-1] [INFO] [1653350927.380549735] [camera.camera]: Set ROS param rgb_camera.profile to default: 640x480x15
[realsense2_camera_node-1] [INFO] [1653350927.389483775] [camera.camera]: Set ROS param gyro_fps to default: 200
[realsense2_camera_node-1] [INFO] [1653350927.390456932] [camera.camera]: Set ROS param accel_fps to default: 100
[realsense2_camera_node-1] [INFO] [1653350927.453666174] [camera.camera]: Stopping Sensor: Stereo Module
[realsense2_camera_node-1] [INFO] [1653350927.537728126] [camera.camera]: Starting Sensor: Stereo Module
[realsense2_camera_node-1]  24/05 00:08:47,823 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:47,974 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1] [INFO] [1653350928.035328885] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 640, Height: 480, FPS: 15
[realsense2_camera_node-1] [INFO] [1653350928.036760077] [camera.camera]: Stopping Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1653350928.057219753] [camera.camera]: Starting Sensor: RGB Camera
[realsense2_camera_node-1] [WARN] [1653350928.071610794] [camera.camera]:
[realsense2_camera_node-1] [INFO] [1653350928.161928209] [camera.camera]: Open profile: stream_type: Color(0), Format: RGB8, Width: 640, Height: 480, FPS: 15
[realsense2_camera_node-1] [INFO] [1653350928.168329198] [camera.camera]: RealSense Node Is Up!
[realsense2_camera_node-1]  24/05 00:08:48,361 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:48,512 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:48,641 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:48,781 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:48,882 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:49,032 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:49,040 ERROR [281471864519072] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130
[realsense2_camera_node-1]  24/05 00:08:49,372 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:49,522 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:49,673 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:49,824 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1]  24/05 00:08:49,975 WARNING [281472334281120] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[realsense2_camera_node-1] [INFO] [1653351263.648198743] [rclcpp]: signal_handler(signal_value=2)
[realsense2_camera_node-1] [INFO] [1653351263.650707678] [camera.camera]: Stop Sensor: Stereo Module
[realsense2_camera_node-1] [INFO] [1653351263.650887736] [camera.camera]: Close Sensor.
[realsense2_camera_node-1] [INFO] [1653351263.784489163] [camera.camera]: Close Sensor - Done.
[realsense2_camera_node-1] [INFO] [1653351263.784805163] [camera.camera]: Stop Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1653351263.784911468] [camera.camera]: Close Sensor.
[realsense2_camera_node-1] [INFO] [1653351263.877262648] [camera.camera]: Close Sensor - Done.
[INFO] [realsense2_camera_node-1]: process has finished cleanly [pid 13711]

@MartyG-RealSense
Copy link
Collaborator

Thanks very much for the log. I note that you are experiencing the warning control_transfer returned error, index: 768, error: Resource temporarily unavailable. This indicates that your Pi may be struggling to keep up with processing the stream of frames arriving from the camera.

A Pi 3B+ only has 1 GB of memory, compared to a greater amount such as 4 GB on a Pi 4. So it is conceivable that the 1 GB of real memory of the Pi is being quickly used up and then it is having to change to using significantly slower memory from the Pi's swapfile (file storage capacity used as virtual memory).

When the pointcloud filter is enabled, are the depth and color topics - /camera/depth/image_rect_raw and /camera/color/image_raw - publishing data even if the pointcloud topic is not?

@Alpaca-kun
Copy link
Author

A Pi 3B+ only has 1 GB of memory, compared to a greater amount such as 4 GB on a Pi 4. So it is conceivable that the 1 GB of real memory of the Pi is being quickly used up and then it is having to change to using significantly slower memory from the Pi's swapfile (file storage capacity used as virtual memory).

I monitored the Raspberry pi 3B+ behavior, via htop, when running the ros2 launch command. During the execution of the ros2 launch command, the CPU usage gets almost 100% temporarily.
But after that, the total memory usage of ROS2 processes and Realsense wrapper stabilizes around 20%.
Could the problem be a lack of processing by the CPU?

When the pointcloud filter is enabled, are the depth and color topics - /camera/depth/image_rect_raw and /camera/color/image_raw - publishing data even if the pointcloud topic is not?

Both on the desktop or raspberry pi, I can check the topics below.

/camera/color/camera_info
/camera/color/image_raw
/camera/color/metadata
/camera/depth/camera_info
/camera/depth/color/points
/camera/depth/image_rect_raw
/camera/depth/metadata
/camera/extrinsics/depth_to_color
/camera/extrinsics/depth_to_depth
/camera/imu
/parameter_events
/rosout
/tf_static

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented May 24, 2022 •
edited
Loading

Sorry, I meant are the topics publishing messages if you use rostopic echo with them, such as:

rostopic echo /camera/depth/image_rect_raw

A Pi 3B is not a strong hardware specification, but you are not using a demanding camera configuration either at 640x480 and 15 FPS.

Even on Pi 4 though there have been occasional problems with pointclouds reported for both ROS1 and ROS2, such as #1970 and #1967 (comment) so it may be an issue related to the Arm processor architecture rather than the specification of your particular Pi model.

Arm-based Nvidia Jetson boards also have pointcloud generation problems in the RealSense ROS wrappers. Typically, the solution for both Pi and Jetson has involved using older librealsense SDK and wrapper versions.

@Alpaca-kun
Copy link
Author

Alpaca-kun commented May 24, 2022 •
edited
Loading

Sorry, I meant are the topics publishing messages if you use rostopic echo with them, such as:
rostopic echo /camera/depth/image_rect_raw

This command also results in a blank screen.

Arm-based Nvidia Jetson boards also have pointcloud generation problems in the RealSense ROS wrappers. Typically, the solution for both Pi and Jetson has involved using older librealsense SDK and wrapper versions.

Hmm... Maybe the use of ROS noetic is recommended then?

Furthermore, I have been searching for solutions and realized that ROS2 performs communication through ports.
Since my company uses a firewall, could the problem be in the restriction of the ports?

@MartyG-RealSense
Copy link
Collaborator

Typically, SDK 2.43.0 and wrapper 2.2.23 are recommended as a ROS1 pairing to downgrade to. The 2.2.23 wrapper is compatible with ROS1 Noetic.

https://github.com/IntelRealSense/realsense-ros/releases/tag/2.2.23

There is no information on the effects of a firewall on the RealSense ROS2 wrapper specifically. A non-RealSense discussion in the link below has a situation that sounds similar to your setup though.

https://answers.ros.org/question/372464/ros2-foxy-cant-discover-nodes-from-other-machines/

@Alpaca-kun
Copy link
Author

Sorry for the delay to reply.

https://answers.ros.org/question/372464/ros2-foxy-cant-discover-nodes-from-other-machines/

I tested most of the commands listed in the link and apparently there is nothing wrong with the communication between the machines:

  • ros2 node list shows me the /camera/camera node on both machines.
  • Multicast communication shows me a message in both directions.
  • ROS_DOMAIN_ID and RMW_IMPLEMENTATION are the same value on both machines.

So, I'll try to follow your advices about downgrade the SDK and wrapper's version with ROS1.
Thank you for your advices!

@MartyG-RealSense
Copy link
Collaborator

Hi @Alpaca-kun Do you have an update about this case that you can provide, please? Thanks!

@MartyG-RealSense
Copy link
Collaborator

Hi @Alpaca-kun Do you require further assistance with this case, please? Thanks!

@Alpaca-kun
Copy link
Author

Sorry for the delay,

I am currently working on another project and I won't be able to give feedback about the issue.
So you can close it.

Thanks for your support!

@MartyG-RealSense
Copy link
Collaborator

You are very welcome, @Alpaca-kun - thanks very much for the update! As you suggest, I will close the issue. Please feel free to re-open it at a future date or create a new issue if you return to this project. Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
D400
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Alpaca-kun @MartyG-RealSense