Fix glm::is_normalized
epsilon test
#1349
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The existing comparison bound of$\epsilon^2$ is improperly scaled for testing an epsilon of the squared vector magnitude. Let $\epsilon$ be our specified epsilon and $\delta$ be the permissible delta of the squared magnitude. Thus, for a nearly-normalized vector, we have
Since we only care about small epsilon, we can assume that$\epsilon^2$ is small and just use $\delta = 2\epsilon$ . And in fact, this is the bound used by GLM (MIT license) ... except they're using
length
and notlength2
for some reason.If we stick an epsilon of$\sqrt{\texttt{epsilon}^2 + 1} - 1$ . This patch makes the effective epsilon $\sqrt{2\times\texttt{epsilon} + 1} - 1$ which still isn't perfect, but it's effectively linear in the domain we care about, only really making a practical difference above an epsilon of 10%.
1.0e-6
into the current implementation, this gives us a computed delta of1.0e-12
: smaller than thef32
machine epsilon, and thus no different than direct float comparison without epsilon. This also gives an effetive epsilon of5.0e-13
; much less than the intended1.0e-6
of intended permitted slack! By doing a bit more algebra, we can find the effective epsilon isTL;DR: the existing
is_normalized
considers a vector normalized if the squared magnitude is withinepsilon*epsilon
of1
. This is wrong and it should be testing if it's within2*epsilon
. This PR fixes it.For absence of doubt, a comparison epsilon of$\texttt{epsilon}^2$ is correct when comparing squared magnitude against zero, such as when testing if a displacement vector is nearly zero.