Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gz-sim: 3k_shapes.sdf #1576

Closed
azeey-test opened this issue Aug 29, 2024 · 15 comments · Fixed by gazebosim/gz-sim#2596
Closed

gz-sim: 3k_shapes.sdf #1576

azeey-test opened this issue Aug 29, 2024 · 15 comments · Fixed by gazebosim/gz-sim#2596

Comments

@azeey-test
Copy link
Collaborator

Configuration

  • os: Any

Links

Process

Step 1 - Instructions work

Step 2 - Images (if there are any) match the result

@bperseghetti
Copy link
Member

I'm available to test.

@Mechazo11
Copy link

  • Test enviornment

    CPU: Ryzen 5 5600x, Ram: 32GB @ 3200 Mhz, GPU: RTX 3080
    Clean copy of Ubuntu 24.04 Humble
    ROS 2 Jazzy Jalisco Full Desktop 
    
  • Summary

    • Over 15GB RAM usage, no shapes generated. Could not get to generate shapes. The suggested command erb -T 1 shapes_population.sdf.erb > shapes_population.sdf shows the following error
    az-ubuntu-2204@az-dev-ubuntu-2404:~$ erb -T 1 shapes_population.sdf.erb > shapes_population.sdf
    /usr/lib/ruby/gems/3.2.0/gems/erb-4.0.2/libexec/erb:120:in `set_encoding': No such file or directory @ rb_sysopen - shapes_population.sdf.erb (Errno::ENOENT)
        from /usr/lib/ruby/gems/3.2.0/gems/erb-4.0.2/libexec/erb:120:in `run'
        from /usr/lib/ruby/gems/3.2.0/gems/erb-4.0.2/libexec/erb:164:in `<top (required)>'
        from /usr/bin/erb:25:in `load'
        from /usr/bin/erb:25:in `<main>'
    
  • Issues

    • Failed to generate shapes, UI got laggy and no asset generated on the UI window. Suggested command throws an error message.
  • Fixes

    • N/A
  • Comments/Suggestions

    • N/A

@arjo129
Copy link

arjo129 commented Sep 3, 2024

  • Test enviornment

    CPU: Ryzen 5 5600x, Ram: 32GB @ 3200 Mhz, GPU: RTX 3080
    Clean copy of Ubuntu 24.04 Humble
    ROS 2 Jazzy Jalisco Full Desktop 
    
  • Summary

    • Over 15GB RAM usage, no shapes generated. Could not get to generate shapes. The suggested command erb -T 1 shapes_population.sdf.erb > shapes_population.sdf shows the following error
    az-ubuntu-2204@az-dev-ubuntu-2404:~$ erb -T 1 shapes_population.sdf.erb > shapes_population.sdf
    /usr/lib/ruby/gems/3.2.0/gems/erb-4.0.2/libexec/erb:120:in `set_encoding': No such file or directory @ rb_sysopen - shapes_population.sdf.erb (Errno::ENOENT)
        from /usr/lib/ruby/gems/3.2.0/gems/erb-4.0.2/libexec/erb:120:in `run'
        from /usr/lib/ruby/gems/3.2.0/gems/erb-4.0.2/libexec/erb:164:in `<top (required)>'
        from /usr/bin/erb:25:in `load'
        from /usr/bin/erb:25:in `<main>'
    
  • Issues

    • Failed to generate shapes, UI got laggy and no asset generated on the UI window. Suggested command throws an error message.
  • Fixes

    • N/A
  • Comments/Suggestions

    • N/A

It doesn't work because you are doing it from your home directory. You need to be in the examples directory for this to work. I'm able to get this working on my machine.

@Mechazo11
Copy link

@arjo129 thank you for the feedback, I will try this tutorial again today (if possible) and report back.

@Mechazo11
Copy link

@arjo129 can you share the steps you took to get the demo working. Mine is crashing. Here are the steps I took

cd ~/workspace/src/gz-sim/examples/worlds
erb -T 1 shapes_population.sdf.erb > shapes_population.sdf
gz sim shapes_population.sdf

A gazebo sim window opens up, RAM usages goes upto 14 GB but nothing else happens. Waited about 3-5 minutes before a message to Force Quit Gazebo sim window popped up.

image

@arjo129
Copy link

arjo129 commented Sep 4, 2024

Can you run with the -v4 option.

@Mechazo11
Copy link

@arjo129 can you give me the full command please. Sorry but kinda very new to Gazebo not familiar with all the command structure yet.

@arjo129
Copy link

arjo129 commented Sep 4, 2024

cd ~/workspace/src/gz-sim/examples/worlds
erb -T 1 shapes_population.sdf.erb > shapes_population.sdf
gz sim -v4 shapes_population.sdf

I can see that the gazebo world takes forever to deserialize.

@Mechazo11
Copy link

Thanks @arjo129 I will try your commands and report back tomorrow

@arjo129
Copy link

arjo129 commented Sep 5, 2024

So I am able to get this working but it needs a lot of RAM. The server starts up relatively quickly, but the gui takes a very long time to load. The server SDF parser takes ~30 seconds to start while the shapes take 10 minutes to appear on the gui screen. Total memory usage is about 10GB of ram and 20GB of swap, I can see why most people would have trouble running this.

arjo129 added a commit to gazebosim/gz-sim that referenced this issue Sep 5, 2024
I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>
@arjo129
Copy link

arjo129 commented Sep 5, 2024

I've opened gazebosim/sdformat#1478. I'm not an expert on libsdformat, but the root cause does seem like this. In the mean time gazebosim/gz-sim#2596 can be used to reduce load times.

@arjo129 arjo129 added the bug Something isn't working label Sep 6, 2024
@utkuesen
Copy link

utkuesen commented Sep 6, 2024

I tested it and it worked for me. However, rendering process is taken so much time.
I tried it on machine which is HP ZBook 15 G3(i7-6820HQ , 32 Gb RAM, NVidia Quadro M1000M, Ubuntu 24.04.1 LTS) without docker .
I started program from home directory with this command "gz sim /usr/share/gz/gz-sim9/worlds/3k_shapes.sdf".
It took at least 5 min to render the all scene.
When i tried with this command "gz sim /usr/share/gz/gz-sim9/worlds/3k_shapes.sdf -v4", I get this warning till scene is created from beginning "(2024-09-06 19:22:46.933) [warning] [GUI] Unable to deserialize sdf::Model".

First minutes of the screen:
image

The final result:
Screenshot from 2024-09-06 19-19-05

@arjo129
Copy link

arjo129 commented Sep 6, 2024

Thanks @utkuesen, yeah the rendering is taking very long. We also know why as can be seen in my previous comments. We just need to come to a consensus on whats the best way forward. The current issue is the gui is taking too long to deserialize the ecm.

@Mechazo11 Mechazo11 removed their assignment Sep 6, 2024
@Mechazo11
Copy link

Sorry, clicked "unassign me" by mistake.

@jennuine
Copy link

jennuine commented Sep 6, 2024

Sorry, clicked "unassign me" by mistake.

I've re-assigned you 😄

@jennuine jennuine assigned jennuine and Mechazo11 and unassigned jennuine Sep 6, 2024
@azeey azeey closed this as completed Sep 16, 2024
arjo129 added a commit to gazebosim/gz-sim that referenced this issue Nov 18, 2024
…ary. (#2596)

* A hack to greatly improve load times

I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Remove magic word

Signed-off-by: Arjo Chakravarty <[email protected]>

* Send empty string instead of using sentinel value.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Style

Signed-off-by: Arjo Chakravarty <[email protected]>

* fix custom sensor system example build (#2649)

Signed-off-by: Ian Chen <[email protected]>

* remove stray change

Signed-off-by: Arjo Chakravarty <[email protected]>

---------

Signed-off-by: Arjo Chakravarty <[email protected]>
Signed-off-by: Ian Chen <[email protected]>
Co-authored-by: Ian Chen <[email protected]>
mergify bot pushed a commit to gazebosim/gz-sim that referenced this issue Nov 19, 2024
…ary. (#2596)

* A hack to greatly improve load times

I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Remove magic word

Signed-off-by: Arjo Chakravarty <[email protected]>

* Send empty string instead of using sentinel value.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Style

Signed-off-by: Arjo Chakravarty <[email protected]>

* fix custom sensor system example build (#2649)

Signed-off-by: Ian Chen <[email protected]>

* remove stray change

Signed-off-by: Arjo Chakravarty <[email protected]>

---------

Signed-off-by: Arjo Chakravarty <[email protected]>
Signed-off-by: Ian Chen <[email protected]>
Co-authored-by: Ian Chen <[email protected]>
(cherry picked from commit 1a88131)
mergify bot pushed a commit to gazebosim/gz-sim that referenced this issue Nov 19, 2024
…ary. (#2596)

* A hack to greatly improve load times

I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Remove magic word

Signed-off-by: Arjo Chakravarty <[email protected]>

* Send empty string instead of using sentinel value.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Style

Signed-off-by: Arjo Chakravarty <[email protected]>

* fix custom sensor system example build (#2649)

Signed-off-by: Ian Chen <[email protected]>

* remove stray change

Signed-off-by: Arjo Chakravarty <[email protected]>

---------

Signed-off-by: Arjo Chakravarty <[email protected]>
Signed-off-by: Ian Chen <[email protected]>
Co-authored-by: Ian Chen <[email protected]>
(cherry picked from commit 1a88131)
mergify bot pushed a commit to gazebosim/gz-sim that referenced this issue Nov 19, 2024
…ary. (#2596)

* A hack to greatly improve load times

I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Remove magic word

Signed-off-by: Arjo Chakravarty <[email protected]>

* Send empty string instead of using sentinel value.

Signed-off-by: Arjo Chakravarty <[email protected]>

* Style

Signed-off-by: Arjo Chakravarty <[email protected]>

* fix custom sensor system example build (#2649)

Signed-off-by: Ian Chen <[email protected]>

* remove stray change

Signed-off-by: Arjo Chakravarty <[email protected]>

---------

Signed-off-by: Arjo Chakravarty <[email protected]>
Signed-off-by: Ian Chen <[email protected]>
Co-authored-by: Ian Chen <[email protected]>
(cherry picked from commit 1a88131)

# Conflicts:
#	include/gz/sim/components/Model.hh
ahcorde pushed a commit to gazebosim/gz-sim that referenced this issue Nov 19, 2024
…ary. (#2596) (#2682)

* A hack to greatly improve load times

I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>
Signed-off-by: Ian Chen <[email protected]>
Co-authored-by: Ian Chen <[email protected]>
(cherry picked from commit 1a88131)
ahcorde pushed a commit to gazebosim/gz-sim that referenced this issue Nov 19, 2024
…ary. (#2596) (#2683)

* A hack to greatly improve load times

I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>
Signed-off-by: Ian Chen <[email protected]>
Co-authored-by: Ian Chen <[email protected]>
(cherry picked from commit 1a88131)
arjo129 added a commit to gazebosim/gz-sim that referenced this issue Nov 20, 2024
…ary. (backport #2596) (#2684)

* Improve load times by skipping serialization of entities when unecessary. (#2596)

* A hack to greatly improve load times

I was investigating
gazebosim/gazebo_test_cases#1576 , in my
investigation it came to my notice that `sdf::Element` takes forever to
destroy (We should open a ticket somewhere about this). If we are
skipping serialization we might as well not create and destroy an
SDF Element. This hack greatly speeds up the load time for gazebo.

Signed-off-by: Arjo Chakravarty <[email protected]>
Co-authored-by: Arjo Chakravarty <[email protected]>
Co-authored-by: Arjo Chakravarty <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants