-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixes precision loss when adding nested models. #314
Conversation
85be3d4
to
dadc373
Compare
dadc373
to
346e688
Compare
addresses #313 |
I think we would need a test before we could merge this. Fortunately, there are some existing tests for the text output after including nested models (see test/integration/nested_model.cc) that could be modified to increase precision. For example, scpeters@dce1c4d increases the precision of a pose value in a nested model and expects the same precision in the output. Is that the type of thing this is supposed to fix? I tried that test with your branch, and it didn't seem to fix it. With which data types have you tested your branch? I think Vector3 and Quaternion values might have precision limited by the ign-math class: |
The proposed fix only addresses double and float types stored in the parameter variant. I ran into this when using camera intrinsics such as fx, fy, etc. I already thought about the vector and matrix types. We need special branches to fix that too as the default streaming operators you pointed to only serialize with 6 decimals precision. I will extend this CL to also fix those types and add a test. |
1fc2107
to
c2e6c16
Compare
The precision loss is resolved by explicitly calling std::setprecision for float and double parameters when these are streamed/requested as strings. Signed-off-by: Hauke Heibel <[email protected]>
c2e6c16
to
bc247b1
Compare
I have modified the existing test to show that the correct code path is taken and to show that the logic is correct. Since I have not found an SDF property with 32bit float precision, I only tested for double precision. I used the sphere's radius property which is a double. All floating point ignition math types are affected as well. I could easily provide full specializations for them as and re-implement the serialization while taking care to exactly copy the math libraries' behavior but I am wondering if the better fix were to remove the |
For the |
There are no global effects because the |
Signed-off-by: Jenn Nguyen <[email protected]>
b71bc4b
to
bc247b1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sweet, thank you for checking!
The precision loss is resolved by explicitly calling std::setprecision
for float and double parameters when these are streamed/requested as
strings.