Invalid plan arising from integer rounding #190
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Issue 1
A lossy conversion between int64 and uint64 can arise because of the use of
float64()
, which has a precision of 53.This issue arises in the following locations:
Taking the form:
Issue 2
Another potential lossy operation can arise from the usage of
big.Float.Set
.This issue arises in the following location:
This is a consequence of the following:
Because
v
uses the default precision of 53 ofbig.Float
An additional consequence of using
target.Set(v)
is that the tests within value_msgpack_test.go would not detect this form of rounding.Fixes
new(big.Float).SetInt64()
ornew(big.Float).SetUint64()
is used as appropriate in value_msgpack.go.big.Float.Copy
instead ofbig.Float.Set
is used within value.go.Related Issues