Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent division by 0 when creating vertices of a PrismMesh #86931

Merged

Conversation

LeoBelda
Copy link
Contributor

@LeoBelda LeoBelda commented Jan 7, 2024

Fixes #86356

When creating a PrismMesh with a height value of 0, a division by 0 occurs in PrismMesh::_create_mesh_array, leading to nan values in vertices, and then to undefined behavior when using the resulting mesh array.

First approach was to simply protect the divisions this way:
float scale = size.y != 0 ? (y - start_pos.y) / size.y : 0
It fixed the crash by preventing nan values to arise, but led to incorrect x coordinates for some of the vertices:
flatprism
This is because we are relying on the current ratio of the y coordinate to compute scale.

Now if we focus on the y variable, we see it is initialized before the loop
y = start_pos.y;
and incremented at the end of the loop for every iteration
y += size.y / (subdivide_h + 1.0);
So we can affirm that at the beginning of the loop, y == start_pos.y + j * (size.y / (subdivide_h + 1.0))

Now back to the scale calculation:
float scale = (y - start_pos.y) / size.y
Let's replace y with the expression we just established and simplify it:

scale = (start_pos.y + j * (size.y / (subdivide_h + 1.0)) - start_pos.y) / size.y
scale = j * (size.y / (subdivide_h + 1.0)) / size.y
scale = j / (subdivide_h + 1.0)

Notice how we got rid of the division that was causing the issue.
Also notice that the variable v2 was already assigned with this operation; I kept it as an alias of scale to keep naming consistent, not sure it is the best choice.

With this solution we have consistent coordinates even when the prism is flat (size.y == 0) :
flatprismfixed

@AThousandShips AThousandShips added bug crash topic:3d cherrypick:4.2 Considered for cherry-picking into a future 4.2.x release labels Jan 7, 2024
@AThousandShips AThousandShips added this to the 4.3 milestone Jan 7, 2024
@AThousandShips AThousandShips requested a review from a team January 7, 2024 16:22
Copy link
Member

@clayjohn clayjohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Tested locally and can confirm that it works.

@LeoBelda LeoBelda force-pushed the fix-create-prism-mesh-division-by-zero branch from 3bfbcb4 to 83992fd Compare January 10, 2024 22:06
@akien-mga akien-mga merged commit 79791a3 into godotengine:master Jan 11, 2024
15 checks passed
@akien-mga
Copy link
Member

Thanks! And congrats for your first merged Godot contribution 🎉

@LeoBelda LeoBelda deleted the fix-create-prism-mesh-division-by-zero branch January 11, 2024 18:22
@YuriSizov YuriSizov removed the cherrypick:4.2 Considered for cherry-picking into a future 4.2.x release label Jan 25, 2024
@YuriSizov
Copy link
Contributor

Cherry-picked for 4.2.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CSGMesh with PrismMesh set to 0 height Crashes
6 participants