Prevent division by 0 when creating vertices of a PrismMesh #86931
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #86356
When creating a PrismMesh with a height value of 0, a division by 0 occurs in
PrismMesh::_create_mesh_array
, leading tonan
values in vertices, and then to undefined behavior when using the resulting mesh array.First approach was to simply protect the divisions this way:
float scale = size.y != 0 ? (y - start_pos.y) / size.y : 0
It fixed the crash by preventing
nan
values to arise, but led to incorrectx
coordinates for some of the vertices:This is because we are relying on the current ratio of the
y
coordinate to computescale
.Now if we focus on the
y
variable, we see it is initialized before the loopy = start_pos.y;
and incremented at the end of the loop for every iteration
y += size.y / (subdivide_h + 1.0);
So we can affirm that at the beginning of the loop,
y == start_pos.y + j * (size.y / (subdivide_h + 1.0))
Now back to the scale calculation:
float scale = (y - start_pos.y) / size.y
Let's replace
y
with the expression we just established and simplify it:Notice how we got rid of the division that was causing the issue.
Also notice that the variable
v2
was already assigned with this operation; I kept it as an alias ofscale
to keep naming consistent, not sure it is the best choice.With this solution we have consistent coordinates even when the prism is flat (size.y == 0) :