Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix included nested model expansion in SDF generation #768

Merged
merged 12 commits into from
May 11, 2021

Conversation

jennuine
Copy link
Contributor

@jennuine jennuine commented Apr 15, 2021

🦟 Bug fix

Part of #479

Summary

This PR resolves the issue of included nested models (e.g., //model/include) being expanded even though the option "Expand include tags" is disabled.

Before PR

  1. Load model_nested_include.sdf in Ignition
  2. Click on the top left menu > Save world as...
  3. Save the world (with "Expand include tags" disabled)
  4. Open the saved sdf file and see the included models are expanded

After PR
Run the above steps again except the saved sdf file will have //include tags instead

Checklist

  • Signed all commits for DCO
  • Added tests
  • Updated documentation (as needed)
  • Updated migration guide (as needed)
  • codecheck passed (See contributing)
  • All tests passed (See test coverage)
  • While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge

@jennuine jennuine added bug Something isn't working 🏢 edifice Ignition Edifice labels Apr 15, 2021
@jennuine jennuine requested a review from azeey April 15, 2021 00:26
@jennuine jennuine requested a review from chapulina as a code owner April 15, 2021 00:26
@jennuine jennuine self-assigned this Apr 15, 2021
@github-actions github-actions bot added the 🏯 fortress Ignition Fortress label Apr 15, 2021
Signed-off-by: Jenn Nguyen <[email protected]>
@codecov
Copy link

codecov bot commented Apr 15, 2021

Codecov Report

Merging #768 (73b7110) into ign-gazebo5 (d3ee64a) will increase coverage by 0.06%.
The diff coverage is 72.72%.

❗ Current head 73b7110 differs from pull request most recent head 32dd57a. Consider uploading reports for the commit 32dd57a to get more accurate results
Impacted file tree graph

@@               Coverage Diff               @@
##           ign-gazebo5     #768      +/-   ##
===============================================
+ Coverage        65.32%   65.39%   +0.06%     
===============================================
  Files              240      240              
  Lines            17624    17679      +55     
===============================================
+ Hits             11513    11561      +48     
- Misses            6111     6118       +7     
Impacted Files Coverage Δ
.../plugins/component_inspector/ComponentInspector.cc 7.04% <0.00%> (-0.17%) ⬇️
src/rendering/RenderUtil.cc 42.10% <0.00%> (-0.05%) ⬇️
src/systems/physics/Physics.cc 69.44% <83.01%> (ø)
src/SdfGenerator.cc 94.77% <89.28%> (-1.23%) ⬇️
src/systems/physics/EntityFeatureMap.hh 93.65% <90.90%> (ø)
src/systems/diff_drive/DiffDrive.cc 86.91% <100.00%> (+0.94%) ⬆️
src/Server.cc 83.43% <0.00%> (+0.63%) ⬆️
src/SimulationRunner.cc 93.77% <0.00%> (+1.03%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a9cfa0d...32dd57a. Read the comment docs.

@chapulina chapulina removed the 🏯 fortress Ignition Fortress label Apr 19, 2021
ASSERT_NE(nullptr, uri);
ASSERT_NE(nullptr, uri->GetText());
EXPECT_EQ(
"https://fuel.ignitionrobotics.org/1.0/OpenRobotics/models/Coke Can",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saving the Fuel version number doesn't seem to work for me. Can you add a test to verify?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modified the previous test, what do you think? 57d5833

if (_saveFuelVersion && uriMapIt != _includeUriMap.end())
{
// find fuel model version from file path
std::string version = common::parentPath(e->FilePath());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is the same as modelDir.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 73b7110

if (isNumber(version))
{
sdf::ElementPtr uriElem = e->GetIncludeElement()->GetElement("uri");
std::string uri = uriElem->GetValue()->GetAsString();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tip: This can be simplified as e->GetIncludeElement()->Get<std::string>("uri")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 481 to 482
_elem->InsertElement(e->GetIncludeElement());
e->RemoveFromParent();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed another thing. This changes the order of models. I'm testing with model_nested_include.sdf and the original has:

      <model name="M2">
        <include>
          <uri>sphere</uri>
          <pose>0 2 2 0 0 0</pose>
        </include>

        <model name="M3">
          <include>
            <name>coke</name>
            <uri>https://fuel.ignitionrobotics.org/1.0/OpenRobotics/models/Coke Can</uri>
            <pose>2 2 2 0 0 0</pose>
          </include>
        </model>
      </model>

The saved file has

      <model name='M2'>
        <model name='M3'>
          <include>
            <name>coke</name>
            <uri>https://fuel.ignitionrobotics.org/1.0/OpenRobotics/models/Coke Can</uri>
            <pose>2 2 2 0 -0 0</pose>
          </include>
        </model>
        <include>
          <uri>sphere</uri>
          <pose>0 2 2 0 -0 0</pose>
        </include>
      </model>

(with <plugin> removed). The order in the original is sphere, M3, whereas in the saved file M3,sphere. The two files will have different results when sdf::Model::ModelByIndex is called. I think can preserve the order if we do this:

Suggested change
_elem->InsertElement(e->GetIncludeElement());
e->RemoveFromParent();
e ->Copy(e->GetIncludeElement());

This is the first time I've used sdf::Element::Copy and I now see its usefulness.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noticing that! 73b7110

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm now getting the `//include/@name attribute set.

      <include name='include_nested_new_name'>
        <uri>include_nested</uri>
        <name>include_nested_new_name</name>
        <pose>1 2 3 0 -0 0</pose>
      </include>

I believe it's because e is a model and has a name attribute and sdf::Element::Copy doesn't seem to be clearing the attributes when copying. I think this is something we should fix in libsdformat. Since I don't know the reason for why attributes are not cleared in https://github.com/osrf/sdformat/blob/25004f0bad227cf11854cc208e74be08e8623359/src/Element.cc#L218-L220, I think it might be best to create another function to clear attributes instead of modifying sdf::Element::Copy.

Copy link
Contributor Author

@jennuine jennuine Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we create both a sdf::Element::RemoveAllAttributes() as well as sdf::Element::RemoveAttribute(string key)? Just to have both options for users

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that sounds good to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the PR that adds these methods: gazebosim/sdformat#555

This will be forward ported to dome & edifice once merged

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new attribute removal methods have been ported to sdf11 and with b5a521c the attributes are cleared before copying the include element

@jennuine jennuine added the needs upstream release Blocked by a release of an upstream library label Apr 30, 2021
@jennuine jennuine mentioned this pull request Apr 30, 2021
7 tasks
@jennuine jennuine removed the needs upstream release Blocked by a release of an upstream library label May 3, 2021
@jennuine
Copy link
Contributor Author

jennuine commented May 3, 2021

@osrf-jenkins run tests please

@jennuine jennuine added the needs upstream release Blocked by a release of an upstream library label May 3, 2021
@jennuine
Copy link
Contributor Author

jennuine commented May 3, 2021

Requires #758 to fix macOS build failure

@jennuine jennuine removed the needs upstream release Blocked by a release of an upstream library label May 5, 2021
@jennuine
Copy link
Contributor Author

jennuine commented May 5, 2021

@osrf-jenkins run tests please

Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks for iterating.

@jennuine jennuine merged commit dca877a into ign-gazebo5 May 11, 2021
@jennuine jennuine deleted the jennuine/sdf_gen branch May 11, 2021 22:49
scpeters added a commit that referenced this pull request May 19, 2021
* 🎈 3.8.0 (#688)

Signed-off-by: Louise Poubel <[email protected]>

* Make it so joint state publisher is quieter (#696)

Signed-off-by: Michael Carroll <[email protected]>

* [BULLET] Making GetContactsFromLastStepFeature optional in Collision Features (#690)

* GetContactsFromLastStepFeature made optional

Signed-off-by: Tomas Lorente <[email protected]>

Co-authored-by: Addisu Z. Taddese <[email protected]>

* Add test for thermal object temperatures below 0 kelvin (#621)

Signed-off-by: Ashton Larkin <[email protected]>

* Scenebroadcaster sensors (#698)

* Add sensors to scene broadcaster

Signed-off-by: Nate Koenig <[email protected]>

* Update src/systems/scene_broadcaster/SceneBroadcaster.cc

Co-authored-by: Michael Carroll <[email protected]>

* Fix codecheck

Signed-off-by: Nate Koenig <[email protected]>

Co-authored-by: Nate Koenig <[email protected]>
Co-authored-by: Michael Carroll <[email protected]>

* Fix diffuse and ambient values for ackermann example (#707)

Signed-off-by: Ammaar Solkar <[email protected]>

* 🎈 5.0.0 (#731)

Signed-off-by: Louise Poubel <[email protected]>

* Support configuring particle scatter ratio in particle emitter system (#674)

* set particle scatter ratio through sdf

Signed-off-by: Ian Chen <[email protected]>

* address feedback

Signed-off-by: Ian Chen <[email protected]>

* add todo note about merging forward

Signed-off-by: Ian Chen <[email protected]>

Co-authored-by: Ashton Larkin <[email protected]>

* Update PlaybackScrubber description (#733)

Signed-off-by: Ammaar Solkar <[email protected]>

* Iterate through changed links only in UpdateSim (#678)

Signed-off-by: Ashton Larkin <[email protected]>

* Do not pass -Wno-unused-parameter to MSVC compiler (#716)

Signed-off-by: Silvio Traversaro <[email protected]>

* Use Protobuf_IMPORT_DIRS instead of PROTOBUF_IMPORT_DIRS for compatibility with Protobuf CMake config (#715)

Signed-off-by: Silvio Traversaro <[email protected]>

* Fix component inspector shutdown crash (#724)

Signed-off-by: Louise Poubel <[email protected]>

Co-authored-by: Alejandro Hernández Cordero <[email protected]>

* Validate step size and RTF parameters (#740)

Only set them if they are strictly positive.

Signed-off-by: Luca Della Vedova <[email protected]>

* Fix compute_rtfs arguments (#737)

Signed-off-by: Caio Amaral <[email protected]>

* Fixed collision visual bounding boxes (#746)

Signed-off-by: Jenn Nguyen <[email protected]>

* Fix CMakelists.txt merge

Signed-off-by: Nate Koenig <[email protected]>

* ECM's ChangedState gets message with modified components (#742)

* ecm's ChangedState to contain modified components

Signed-off-by: Jenn Nguyen <[email protected]>

* updated log_system test

Signed-off-by: Jenn Nguyen <[email protected]>

* removed unnecessary calls

Signed-off-by: Jenn Nguyen <[email protected]>

Co-authored-by: Ian Chen <[email protected]>

* fixed particle emitter forward playback (#745)

Signed-off-by: Jenn Nguyen <[email protected]>

* Merge pull request #730 from ignitionrobotics/particle_emitter

Particle emitter based on SDF

* 4 7 0 prep (#755)

* Prepare for 4.7.0

Signed-off-by: Nate Koenig <[email protected]>

* Added placeholder

Signed-off-by: Nate Koenig <[email protected]>

Co-authored-by: Nate Koenig <[email protected]>

* Fix 'invalid animation update data' msg for actors (#754)

Signed-off-by: Ashton Larkin <[email protected]>

* Update benchmark comparison instructions (#766) (#766)

Signed-off-by: Ashton Larkin <[email protected]>

* [DiffDrive] add enable/disable (#772)

* add enable/disable diffdrive

Signed-off-by: Guillaume Doisy <[email protected]>

* remove debug

Signed-off-by: Guillaume Doisy <[email protected]>

* do not subscribe to enable if topic is empty

Signed-off-by: Guillaume Doisy <[email protected]>

* add test

Signed-off-by: Guillaume Doisy <[email protected]>

* lint and style

Signed-off-by: Guillaume Doisy <[email protected]>

* change enable type to bool and renamed to enabled

Signed-off-by: Guillaume Doisy <[email protected]>

* Add odometry publisher system (#547)

* Create Initial Odometry Publisher system plugin

Add code for initial plugin that gets position from Pose component and
calculates velocities based on rolling mean from displacement data.

Signed-off-by: Maganty Rushyendra <[email protected]>

* Remove Linear and Angular Velocity components

Also renames frames in Odometry msg to include model name, and makes
various style changes.

Signed-off-by: Maganty Rushyendra <[email protected]>

* Get World pose instead of pose of robot base frame

Signed-off-by: Maganty Rushyendra <[email protected]>

* Add documentation for variables and functions

Includes minor stylistic changes.

Signed-off-by: Maganty Rushyendra <[email protected]>

* Check for valid odomTopic and update copyright year

Signed-off-by: Maganty Rushyendra <[email protected]>

* Add tests for OdometryPublisherSystem and fix velocity calculation bug

Swap X and Y linear velocities when calculating odometry velocities
relative to robotBaseFrame.

Signed-off-by: Maganty Rushyendra <[email protected]>

Co-authored-by: ahcorde <[email protected]>

* Patch particle emitter2 service (#777)

* Patch particle emitter2 service

Signed-off-by: Nate Koenig <[email protected]>

* Remove condition variable

Signed-off-by: Nate Koenig <[email protected]>

* Set emitter frame and relative pose

Signed-off-by: Nate Koenig <[email protected]>

Co-authored-by: Nate Koenig <[email protected]>

* Preparing for 4.8.0 release (#780)

Signed-off-by: Nate Koenig <[email protected]>

Co-authored-by: Nate Koenig <[email protected]>

* 👩‍🌾 Enable Focal CI (#646)

Signed-off-by: Louise Poubel <[email protected]>

Co-authored-by: Michael Carroll <[email protected]>

* [TPE] Support setting individual link velocity  (#427)

Signed-off-by: claireyywang <[email protected]>
Signed-off-by: Ian Chen <[email protected]>

Co-authored-by: Ian Chen <[email protected]>
Co-authored-by: Louise Poubel <[email protected]>

* Don't store duplicate ComponentTypeId in ECM (#751)

Signed-off-by: Louise Poubel <[email protected]>

* Feature/hydrodynamics (#749)

Implement hydrodynamics and thruster plugin.

Signed-off-by: Arjo Chakravarty <[email protected]>
Co-authored-by: Mabel Zhang <[email protected]>
Co-authored-by: Carlos Agüero <[email protected]>

* Fix macOS build: components::Name in benchmark (#784)

Signed-off-by: Louise Poubel <[email protected]>

Co-authored-by: Steve Peters <[email protected]>

* Fix ColladaExporter submesh index bug (#763)

Signed-off-by: Jorge Perez <[email protected]>

* 👩‍🌾 Fix Windows build and some warnings (#782)

Signed-off-by: Louise Poubel <[email protected]>

Co-authored-by: Alejandro Hernández Cordero <[email protected]>

* Prevent crash on Plotting plugin with mutex (#747)

Signed-off-by: Louise Poubel <[email protected]>

* Bump ign-physics version to 3.2 (#792)

Signed-off-by: Louise Poubel <[email protected]>

* Bump to ign-msgs 7.1 / sdformat 11.1, Windows fixes (#758)

Signed-off-by: Louise Poubel <[email protected]>

* Util: Use public API from libsdformat for detecting non-file source (#794)

Signed-off-by: Eric Cousineau <[email protected]>

* Fix included nested model expansion in SDF generation (#768)

* fixed included nested model expansion

Signed-off-by: Jenn Nguyen <[email protected]>

* added resource path to test

Signed-off-by: Jenn Nguyen <[email protected]>

* use orig URIs & support multi level nesting

Signed-off-by: Jenn Nguyen <[email protected]>

* save fuel version when enabled

Signed-off-by: Jenn Nguyen <[email protected]>

* retrieve uri from map

Signed-off-by: Jenn Nguyen <[email protected]>

* copy included element

Signed-off-by: Jenn Nguyen <[email protected]>

* clear attributes before copying include element

Signed-off-by: Jenn Nguyen <[email protected]>

* Map canonical links to their models (#736)

Signed-off-by: Ashton Larkin <[email protected]>

* ColladaExporter, export submesh selected (#802)

* Export only submesh if selected
* Add test case for the PR
* Attempting a unified solution

Signed-off-by: Jorge Perez <[email protected]>
Co-authored-by: Nate Koenig <[email protected]>

Co-authored-by: Michael Carroll <[email protected]>
Co-authored-by: Jose Tomas Lorente <[email protected]>
Co-authored-by: Addisu Z. Taddese <[email protected]>
Co-authored-by: Ashton Larkin <[email protected]>
Co-authored-by: Nate Koenig <[email protected]>
Co-authored-by: Nate Koenig <[email protected]>
Co-authored-by: Ammaar Solkar <[email protected]>
Co-authored-by: Ian Chen <[email protected]>
Co-authored-by: Ashton Larkin <[email protected]>
Co-authored-by: Silvio Traversaro <[email protected]>
Co-authored-by: Alejandro Hernández Cordero <[email protected]>
Co-authored-by: Luca Della Vedova <[email protected]>
Co-authored-by: Caio Amaral <[email protected]>
Co-authored-by: Jenn Nguyen <[email protected]>
Co-authored-by: G.Doisy <[email protected]>
Co-authored-by: Rushyendra Maganty <[email protected]>
Co-authored-by: Claire Wang <[email protected]>
Co-authored-by: Arjo Chakravarty <[email protected]>
Co-authored-by: Mabel Zhang <[email protected]>
Co-authored-by: Carlos Agüero <[email protected]>
Co-authored-by: Steve Peters <[email protected]>
Co-authored-by: Jorge Perez <[email protected]>
Co-authored-by: Eric Cousineau <[email protected]>
Co-authored-by: Jorge Perez <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working 🏢 edifice Ignition Edifice
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants