Unable to move/rotate shapes (reliably)

[dignakov]

I'm having trouble moving/rotating shapes with Transform Control.

If I just click on the arrows they mostly don't engage and I end up moving the camera around the scene. Occasionally and incorrect arrow (direction) engages. (https://youtu.be/WbWLZBDpXgQ)

Translation seems to work if I hold x/y/z, but rotation still does not.

I'm running Ubuntu 18.04 docker image (Ubuntu 19.10 host). I compiled ignition gazebo today from source (3.1.0). I also tried installing binary (3.0.0), and I can't move the shapes at all in that version.

[chapulina]

Thank you for the bug report and the video. I can't reproduce the issue locally either in a Ubuntu Bionic host or inside a Docker container.

My guess is that the mouse position is not being properly mapped to the 3D scene due to some resolution mismatch.

[dignakov]

Is there anything else I can do to help?

The mouse position seems to be ok when selecting shapes (https://youtu.be/Uzj_vx-EwzA), and when interacting with the sidebar UI.

Gazebo (classic) also runs ok inside the same docker container. I can add models move them etc. Other ROS GUI tools like Rviz, and rqt seem to work as well. I can make a few videos if you'd like.

I wanted to compile ignition gazebo on 19.10 without docker, but Ogre seems to build and install some dependencies from source that might conflict with what I'm already using for some other development, I can't risk it right now...

I will try to compile again on 20.04 once ignition gazebo and it's dependencies are available.

[chapulina]

I will try to compile again on 20.04 once ignition gazebo and it's dependencies are available.

I just tried debs inside a 20.04 Docker container and I can't reproduce the issue there either (physics is not working yet though, we need a new release with gazebo-release/gz-physics2-release#4).

The mouse position seems to be ok when selecting shapes

Interesting, that does seem to indicate that the mouse positions are hitting the correct place on the scene.

Is there anything else I can do to help?

I can't think of anything else. Maybe @iche033 has some ideas?

[iche033]

I just tested this but was unable to reproduce either.

@EvilPictureBook what graphics card and driver are you using? Do you have a high dpi display?

[dignakov]

I'm using a 4k monitor, with Nvidia 1060, proprietary driver version: 440.82 from http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu.

Turns out I have an old laptop with 18.04 around, so I'll try installing on that tomorrow too.

If this is a Docker or my config issue and not a bug in Gazebo I don't want to take up your time with it, but just in case it's helpful this is my Dockerfile:

FROM osrf/ros:melodic-desktop-full
RUN apt-get clean && apt-get -y update && apt-get install -y locales && locale-gen en_US.UTF-8
RUN apt-get install -y software-properties-common && add-apt-repository -y ppa:kelleyk/emacs

ENV LANG en_US.UTF-8
ENV TERM xterm-256color

ENV NVIDIA_VISIBLE_DEVICES \
    ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES \
    ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics

RUN apt-get update && apt-get install -y \
        ros-melodic-ros-control \
        ros-melodic-ros-controllers \
        python3-vcstool \
        git bash-completion \
        openssh-server \
        locales \
	tmux \
	curl \
	wget \
        silversearcher-ag \
        iputils-ping \
        usbutils \
        htop \
	vim-gtk3 \
        tree \
        ranger \
        emacs26 \
	sudo \
	libgl1-mesa-glx \
	libgl1-mesa-dri \
	mesa-utils \
	unzip \
        python3 \
        python3-numpy \
        python3-dev \
        python3-pip \
        python3-catkin-pkg-modules \
        python3-colcon-common-extensions \
        python-catkin-tools \
        clang \
        libclang-dev \
        llvm
        #&& rm -rf /var/likb/apt/lists/*

RUN pip3 install numpy virtualenv jupyterlab rospkg 'python-language-server[all]'
CMD ["bash"]

I'm launching it with...:

xhost +local:
docker run -t -d --network host \
  --user=$(id -u) \
  -e DISPLAY=$DISPLAY \
  -e QT_GRAPHICSSYSTEM=native \
  -e QT_X11_NO_MITSHM=1 \
  -e CONTAINER_NAME=ros-melodic-dev \
  -e USER=$USER \
  --workdir=/home/$USER \
  -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" --privileged \
  -v "/etc/group:/etc/group:ro" \
  -v "/etc/passwd:/etc/passwd:ro" \
  -v "/etc/shadow:/etc/shadow:ro" \
  -v "/etc/sudoers.d:/etc/sudoers.d:ro" \
  -v "/etc/udev/rules.d:/etc/udev/rules.d" \
  -v "/home/$USER/:/home/$USER/" \
  -v "/dev:/dev" --privileged \
  --runtime=nvidia \
  --name=ros-melodic-dev \
  melodic:dev

[dignakov]

I was able to install and use the binary distribution on regular 18.04 without docker (haven't tried compiling from source yet). So it's probably something to do with docker and GUI apps or my config. If I figure it out I'll post an update.

Thank you.

[dignakov]

Sorry for the delay. I finally have a bit more time, and I'd like to test this on 20.04, and close the issue if everything works (since then it's probably a docker issue).

So I was wondering if it's possible yet to install gazebo on 20.04 (if this should go into a separate issue I'd be happy to start one).

(1)
For the binary install, the instructions on the README only has instructions for Bionic. There seem to be binaries available for Focal but they're libignition-gazebo3-dev when the README says to install libignition-gazebo4-dev.

(2)
When trying to install from source some of the dependencies seem to not be available for Focal yet.

EDIT:
I think I'm doing all of this wrong . . . the citadel docs also have colcon instructions which seem to make more sence, but libdart6-dev and a few others aren't available...

I think I will wait for an update to install instructions for 20.04 . . . and close this issue since it seems to work outside of docker on 18.04

Thanks for all the help! Looking forward to trying this on 20.04.

[dignakov]

@chapulina Sorry since this is ROS related I'm not sure if this is the right place to post this issue, but since it's so similar to the one I started with I thought I'd start here. If this is not the right place for it, please let me know and I'll move it.

I tried the ign gazebo release that comes with ROS Noetic and I'm seeing very similar issues: https://youtu.be/BxEcnsSzNlo

I installed: apt install ros-noetic-ros-ign
And ran it with: rosrun ros_ign_gazebo ign_gazebo shapes.sdf

rosrun  ros_ign_gazebo ign_gazebo --version

Ignition Gazebo, version 3.0.0
Copyright (C) 2018 Open Source Robotics Foundation.
Released under the Apache 2.0 License.

[chapulina]

I'm not sure if this is the right place to post this issue

It looks like although you're going through ROS, the issue is still on Ignition, so this should be the right place.

I still don't have any tips on how to fix it though.

[dignakov]

Is there any way I can help more? Any information (driver/system) / debug output / or for me to try to compile anything and see how it goes, I'm happy to try it.

Also if there are any new instructions for building from source on 20.04, I can do that and post if the issue persists.

[chapulina]

It sounds like it's not a high DPI issue, because you're being able to select the entities correctly. Have you tried clicking all around the arrows to see if they're selected if you click slightly outside them?

I'm not sure how else to debug it besides having you compile ign-gazebo from source and put some print outs on these lines to see why the gizmo is not being selected:

https://github.com/ignitionrobotics/ign-gazebo/blob/8ec8d66cc8a6b661a2db6ccae153a785287fa315/src/gui/plugins/scene3d/Scene3D.cc#L1057-L1065

[chapulina]

@EvilPictureBook , you may want to give #291 a try just in case it's a DPI issue.

[dignakov]

Sorry for another long delay (and an update to my account name).

Tested today with the ros2 release, and compiling from source on 20.04. (No Docker, 4k, Nvidia proprietary 450 drivers)

The issue persists. I tried enabling/disabling fractional scaling, and setting QT_AUTO_SCREEN_SCALE_FACTOR to various values.

I added print outs to the section of code you suggested. The reported axis seems to always be [0,0,0], and it never makes it to the print (4).

Video: https://youtu.be/Pt9_yrCSx3M

My section of code:

      // get the visual at mouse position
      rendering::VisualPtr visual = this->dataPtr->camera->VisualAt(
            this->dataPtr->mouseEvent.PressPos());
	  std::cout<<"[=========DEBUG==========]: (1) (visual) "<<visual<<std::endl;
	
      if (visual)
      {
	  std::cout<<"[=========DEBUG==========]: (2) (visual) "<<visual<<std::endl;
        // check if the visual is an axis in the gizmo visual
        math::Vector3d axis =
            this->dataPtr->transformControl.AxisById(visual->Id());
	  	std::cout<<"[=========DEBUG==========]: (3) (axis) "<<axis<<std::endl;
        if (axis != ignition::math::Vector3d::Zero)
        {
	  	  std::cout<<"[=========DEBUG==========]: (4) (axis) "<<axis<<std::endl;
          // start the transform process
          this->dataPtr->transformControl.SetActiveAxis(axis);
          this->dataPtr->transformControl.Start();
          this->dataPtr->mouseDirty = false;
        }
        else
          return;
      }

Print outs:

[=========DEBUG==========]: (1) (visual) 0x7f42cc827fc0
[=========DEBUG==========]: (2) (visual) 0x7f42cc827fc0
[=========DEBUG==========]: (3) (axis) 0 0 0
[=========DEBUG==========]: (1) (visual) 0x7f42cc827fc0
[=========DEBUG==========]: (2) (visual) 0x7f42cc827fc0
[=========DEBUG==========]: (3) (axis) 0 0 0
[=========DEBUG==========]: (1) (visual) 0x7f42cc827fc0
[=========DEBUG==========]: (2) (visual) 0x7f42cc827fc0
[=========DEBUG==========]: (3) (axis) 0 0 0
.
.
.

[iche033]

I remember @jennuine mentioned that fractional scaling has to be enabled and the scaling factor needs to be at 100% for things to work properly.

Just checking, do you have more than one monitor with different size / resolution plugged in?

[dignakov]

Just one monitor. I had fractional scaling on at 100% for a few reboots now with no changes.

[ahcorde]

@dignakov Do you mind to try this PR gazebosim/gz-rendering#221 ?

[dignakov]

@ahcorde Thanks for the link, and sorry for not getting to it sooner.

The PR does not seem to fix my issue. It still acts as in the videos from before.

I compiled using the instructions here: https://ignitionrobotics.org/docs/edifice/install_ubuntu_src, with PR gazebosim/gz-rendering#221 checked out for ign-rendering

In case this is useful:

$ vcs status
................
=== ./ign-cmake (git) ===
On branch ign-cmake2
Your branch is up to date with 'origin/ign-cmake2'.

nothing to commit, working tree clean
=== ./ign-common (git) ===
On branch ign-common3
Your branch is up to date with 'origin/ign-common3'.

nothing to commit, working tree clean
=== ./ign-fuel-tools (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-gazebo (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-gui (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-launch (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-math (git) ===
On branch ign-math6
Your branch is up to date with 'origin/ign-math6'.

nothing to commit, working tree clean
=== ./ign-msgs (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-physics (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-plugin (git) ===
On branch ign-plugin1
Your branch is up to date with 'origin/ign-plugin1'.

nothing to commit, working tree clean
=== ./ign-rendering (git) ===
On branch fixed_transform_control
nothing to commit, working tree clean
=== ./ign-sensors (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-tools (git) ===
On branch ign-tools1
Your branch is up to date with 'origin/ign-tools1'.

nothing to commit, working tree clean
=== ./ign-transport (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./ign-utils (git) ===
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
=== ./sdformat (git) ===
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

[BigMassive]

Unable to move / rotate issue reproduced as per the @dignakov original.
Ignition Edifice binary installed on 20.04 (for a first look at Ignition).

Following the shapes tutorial, on attempting to translate an object, moving a model only works if one of x,y,z is constrained (key pressed). Otherwise, the mouse pointer just moves the camera view of the world.

I cannot get a model to rotate - even when keys are pressed.

[The simulation then seems to fail depending on which model is selected by the transform tool - I think this might be a separate issue]

[dignakov]

I'm still seeing the same issue on Fortress (on my main computer as above, no docker no virtualization, proprietary nvidia drivers), and on a laptop running with nvidia RTX 2060 proprietary drivers as well.

$ nvidia-smi
Sun Oct  3 20:59:25 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.63.01    Driver Version: 470.63.01    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   47C    P8     4W /  N/A |    362MiB /  5934MiB |      3%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

Both systems are Ubuntu 20.04, both with proprietary nvidia drivers. Laptop has a 1080p monitor (unlike the desktop which has a 4k monitor since I know that came up, and I did try the fixes recommended above).

If I can help debug this please let me know what else I can do. I really would like to switch from Gazebo Classic to Ignition, but I can't seem to get it to run.

[iche033]

@dignakov, can you give this patch a try?

diff --git a/ogre2/src/Ogre2Camera.cc b/ogre2/src/Ogre2Camera.cc
index 86d1d7b9..b5043a52 100644
--- a/ogre2/src/Ogre2Camera.cc
+++ b/ogre2/src/Ogre2Camera.cc
@@ -259,7 +259,7 @@ VisualPtr Ogre2Camera::VisualAt(const ignition::math::Vector2i &_mousePos)
       this->ImageWidth(), this->ImageHeight());
   }
 
-  float ratio = screenScalingFactor();
+  float ratio = 1.0f;
   ignition::math::Vector2i mousePos(
       static_cast<int>(std::rint(ratio * _mousePos.X())),
       static_cast<int>(std::rint(ratio * _mousePos.Y())));

[clydemcqueen]

I have noticed that I can consistently grab the rotation or translation handles (arrows, rings) iff I mouse down on a spot where the handle is drawn on top of some other object -- any other object -- and the handle appears in a solid color. I can't grab the parts of the handle that are semi-transparent. I'm not sure if this is intended.

I am running the latest stable Garden built from source on a 20.04 box. I am not running in Docker.

$ nvidia-smi
Sat Mar 12 11:36:46 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.103.01   Driver Version: 470.103.01   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro K1200        Off  | 00000000:01:00.0 Off |                  N/A |
| 79%   80C    P0     6W /  35W |   1819MiB /  4043MiB |     39%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

[azeey]

I'm having the same issue. As shown in the video, I'm not able to select the transform tool if there is no object in the background.

Kazam_screencast_00015.mp4

I'm building garden from source.

[ejalaa12]

I have the same issue, and it also happens when using the rotate mode.
Is there any update on this issue?

[iche033]

I just tested on Garden and the issue shown in the video in #147 (comment) still exists.

[iche033]

I'm having the same issue. As shown in the video, I'm not able to select the transform tool if there is no object in the background.

should be fixed by gazebosim/gz-rendering#800

[iche033]

A couple of PRs addressing mouse interaction issues were merged:

• Fix selecting semi-transparent visuals gz-rendering#800 (Garden)
• Fix transparency over heightmap gz-rendering#811 (Fortress and forward ported to Garden)

Please re-open or create a new issue if you're still facing mouse translation/rotation issues.