Add Jolt Physics as an alternative 3D physics engine

[mihe]

Closes godotengine/godot-proposals#7308.

This pull request adds a new engine module (found under modules/jolt_physics) which integrates Jolt Physics, a 3D physics engine developed by Jorrit Rouwe (@jrouwe) with a focus on games and VR applications, used in titles such as Horizon Forbidden West.

Users of this new module should (generally, but not always) be able to expect better physics performance, both in terms of CPU performance and simulation stability, when compared to Godot Physics.

This new module is a port of the Godot Jolt extension, developed by myself over the past two years, in large part as a personal project, but with the last year (including this port) being sponsored by W4 Games. I've also received a significant amount of assistance and contributions from @jrouwe during this development, which I'm very grateful for. In fact, several major features have been added to Jolt largely to cater to the needs of Godot.

Warning

Note that while this code has gone through a fair amount of battle-testing as part of the Godot Jolt extension there have been non-trivial refactorings done as part of this port that have not been tested much at all. As such, this module should be considered experimental for now. This module is also not at feature parity with Godot Physics just yet, nor at feature parity with the Godot Jolt extension either.

This does not replace Godot Physics as the default 3D physics engine. You will need to opt in by setting the physics/3d/physics_engine project setting to Jolt Physics.

Notable differences to Godot Physics

While the goal is to have this new module be a drop-in replacement for Godot Physics (within reason), Jolt inherently differs from it in a number of ways, which means that it is unlikely that you will see the exact same behavior when switching between the two. There are also several higher-level decisions taken within the module that deviate further from how Godot Physics does things.

Area3D and static bodies

[Click to expand/collapse]

When using Jolt, Area3D will not detect overlaps with StaticBody3D (nor a RigidBody3D frozen with FREEZE_MODE_STATIC) by default, for performance reasons. If you have many/large Area3D overlapping with complex static geometry, such as ConcavePolygonShape3D or HeightMapShape3D, you can end up wasting a significant amount of CPU performance without realizing it.

For this reason this behavior is opt-in through the project setting physics/jolt_physics_3d/simulation/areas_detect_static_bodies, with the recommendation that you set up your collision layers/masks in such a way that only the relevant Area3D are able to detect collisions with static bodies.

This should probably be made into an Area3D property instead, through something like what's been suggested here.

Joint properties

[Click to expand/collapse]

The current interfaces for the 3D joint nodes, which seem to be derived from the Bullet Physics library, don't quite line up with the interface of Jolt's own joints. As such, there are a number of joint properties that are not supported, mainly ones related to configuring the joint's soft limits.

The unsupported properties are:

• PinJoint3D: bias, damping, impulse_clamp
• HingeJoint3D: bias, softness, relaxation
• SliderJoint3D: angular_*, *_limit/softness, *_limit/restitution, *_limit/damping
• ConeTwistJoint3D: bias, relaxation, softness
• Generic6DOFJoint3D: *_limit_*/softness, *_limit_*/restitution, *_limit_*/damping, *_limit_*/erp

Currently an error is emitted if you set these properties to anything but their default values.

In the Godot Jolt extension I exposed alternative joint nodes that better matched Jolt's interface, but these have not been included in this module. Instead, the above mentioned properties should probably be removed and replaced by properties more fitting for Jolt, through something like what's been suggested here.

Single-body joints

[Click to expand/collapse]

You can, in Godot, omit one of the joint bodies for a two-body joint and effectively have "the world" be the other body. However, the node path that you assign your body to (node_a vs node_b) is ignored. Godot Physics will always behave as if you assigned it to node_a, and since node_a is also what defines the frame of reference for the joint limits, you end up with inverted limits and a potentially strange limit shape, especially if your limits allow both linear and angular degrees of freedom.

This is arguably a bug and should be changed. This is also not how single-body joints behaves with Bullet in Godot 3, which behaves as if you assigned the body to node_b instead. It's not clear why this changed in Godot 4.

For this reason this module will, like Bullet in Godot 3, behave as if you assigned the body to node_b instead, with node_a representing "the world".

There is a project setting called physics/jolt_physics_3d/joints/world_node that lets you toggle this behavior, if you need compatibility for an existing project.

Collision margins

[Click to expand/collapse]

Jolt (and other similar physics engines) uses something that Jolt refers to as "convex radius" to help improve the performance and behavior of the types of collision detection that Jolt relies on for convex shapes. Other physics engines (Godot included) might refer to these as "collision margins" instead. Godot exposes these as the margin property on every Shape3D-derived class, as a leftover from the Bullet integration in Godot 3, but Godot Physics itself does not use them for anything.

What these collision margins sometimes do in other engines (as described in Godot's documentation) is effectively add a "shell" around the shape, slightly increasing its size while also rounding off any edges/corners. In Jolt however, these margins are first used to shrink the shape, and then the "shell" is applied, resulting in edges/corners being similarly rounded off, but without increasing the size of the shape.

To prevent having to tweak this margin property manually, since its default value can be problematic for smaller shapes, this module exposes a project setting called physics/jolt_physics_3d/collisions/collision_margin_fraction which is multiplied with the smallest axis of the shape's AABB to calculate the actual margin. The margin property of the shape is then instead used as an upper bound.

These margins should, for most use-cases, be more or less transparent, but can sometimes result in odd collision normals when performing shape queries. You can lower the above mentioned project setting to mitigate some of this, including setting it to 0, but too small of a margin can also cause odd collision results, so is generally not recommended.

Compound shapes

[Click to expand/collapse]

Compound shapes (i.e. bodies with more than one shape) are implemented differently from Godot Physics.

Jolt offers the choice of two compound shapes, one called MutableCompoundShape and one called StaticCompoundShape. The former trades in runtime performance for faster construction/modification time, and vice versa for the latter.

Godot Physics maps closer to Jolt's MutableCompoundShape, but I decided to go with StaticCompoundShape for this implementation, as it simplified things a bit (with being able to discard and rebuild the whole thing when modified) and I figured more people would benefit from the improved runtime performance as opposed to mutation performance.

To mitigate the cost of this potentially expensive rebuild, I made it so that shape changes are only ever "committed" to Jolt when the body has entered into a scene tree. This means that you can make shape changes (including adding/removing shapes) very quickly so long as the body isn't attached to the scene tree. However, if you do in fact add the body to a scene tree, and then start adding/removing/changing shapes on the body, you can end up with worse performance than Godot Physics. This is prominently visible in the "Voxel Game" demo project, as one example.

The plan I have in mind for this (discussed in jrouwe/JoltPhysics#1165) is to replace StaticCompoundShape with a custom compound shape that wraps StaticCompoundShape, but which will always defer its building/committing only until absolutely necessary, meaning either right before a simulation step or when performing queries against the body.

If the use of StaticCompoundShape within this new custom compound shape still ends up being a problem for some use-cases it would likely be trivial to expose some setting or property that lets you use MutableCompoundShape instead.

Scaling shapes/bodies/queries

[Click to expand/collapse]

Godot Physics supports scaling the transform of collision shapes, shape queries, as well as static and kinematic bodies, meaning StaticBody3D, CharacterBody3D, AnimatableBody3D and frozen RigidBody3D. It does not however support scaling simulated/dynamic bodies, such as a non-frozen RigidBody3D, and will effectively discard any such scaling, instead treating it as (1, 1, 1).

Jolt does however support scaling everywhere, and I've tried my best to utilize that, which means that RigidBody3D will support scaling when using Jolt, despite the warning shown on the node currently.

Jolt also supports non-uniform scaling, so long as the inherent primitive shape is preserved. For example, you can scale a cylinder along its height axis but not along its other axes. You will however currently see warnings on the shape node when doing this, due to Godot Physics not supporting this.

Since invalid scaling can cause a number of weird artifacts, and sometimes outright crash the simulation, there are runtime error checks that "sanitize" all scaling to be valid for that particular shape arrangement, and then reports an error if the difference is above an arbitrary threshold. These errors have however proven to be a bit frustrating for some users, who might not care about the sometimes minor corrections that this error-checking applies, so this likely needs a different approach.

Shape-casting

[Click to expand/collapse]

Due to Godot having a "safe" and "unsafe" fraction in the results of cast_motion (and ShapeCast3D), meaning the distances at which the cast shape was found to be colliding and not colliding respectively, it was not viable to rely on Jolt's own shape-casting (which uses conservative advancement) to implement cast_motion. Instead cast_motion is implemented with a binary search, similar to how Godot Physics does it.

However, in Godot Physics this binary search is hardcoded to 8 steps, which tends to result in quite jittery output over even moderate distances. With Godot Jolt (and consequently this module) I chose to instead dynamically calculate the number of steps based on the cast distance, aiming for roughly millimeter precision, and then clamp it between 4 and 16 steps, which seems to have worked out well.

This does however mean that cast_motion will technically, when using Jolt, cost more CPU performance the farther you cast the shape.

Note that this also applies to body_test_motion, and consequently test_move, move_and_collide and move_and_slide.

Baumgarte stabilization

[Click to expand/collapse]

Jolt employs a technique in its solver called Baumgarte stabilization, which is meant to mitigate constraint drift within the simulation, resulting in a more stable simulation. This technique can however result in some artifacts, like piles of bodies not separating as quickly as one might expect.

The strength of this stabilization can be tweaked using the project setting physics/jolt_physics_3d/simulation/baumgarte_stabilization_factor. Setting this project setting to 1.0 will effectively disable the technique, resulting in a simulation that more closely resembles Godot Physics, but which is also more unstable.

Motion queries (move_and_slide, etc.)

[Click to expand/collapse]

Physics servers in Godot are meant to implement the PhysicsServer3D.body_test_motion method, which in turn powers methods like PhysicsBody3D.test_move, PhysicsBody3D.move_and_collide and CharacterBody3D.move_and_slide, which are largely meant to be used for moving player characters around.

PhysicsServer3D.body_test_motion is split into three parts, the first being depenetration (called "recovery" in Godot), the second being a shape-cast from the "recovered" position, and the third being the actual collision check, at the "unsafe" fraction of the shape-cast.

The "recovery" step in Godot Physics is hardcoded to always do 4 iterations with 40% depenetration per iteration. I figured these constants might be useful to expose, so when using this module they can be configured in the project settings as physics/jolt_physics_3d/motion_queries/recovery_iterations and physics/jolt_physics_3d/motion_queries/recovery_amount respectively.

The implementation of PhysicsServer3D.body_test_motion in this module also differs slightly from the Godot Physics implementation, as replicating the Godot Physics version resulted in a prohibitive amount of ghost collisions for move_and_slide.

The discrepancies are as follows:

1. There is no TEST_MOTION_MIN_CONTACT_DEPTH_FACTOR.
2. There is no epsilon added to the safe margin during recovery.
3. Contacts with normals that are not opposing the motion vector are discarded from the final collision check.

While these discrepancies do seem to result in less ghost collisions when using Jolt, they also introduce new problems:

1. move_and_slide is generally slower than in Godot Physics, due to CharacterBody3D::_snap_on_floor triggering on every call.
2. CharacterBody3D will not report wall collisions unless you're moving towards them.

It's not clear to me how to fix these issues, but I suspect they will require changes to move_and_slide on the scene side of things.

Ghost collisions

[Click to expand/collapse]

Jolt employs two techniques to mitigate ghost collisions, meaning collisions with internal edges of shapes/bodies.

The first technique, called "active edge detection", marks edges of triangles in ConcavePolygonShape3D or HeightMapShape3D as either "active" or "inactive", based on the angle to the neighboring triangle. When a collision happens with an inactive edge the collision normal will be replaced with the triangle's normal instead, to lessen the effect of ghost collisions.

The angle threshold for this active edge detection is configurable through the project setting physics/jolt_physics_3d/collisions/active_edge_threshold.

The second technique, called "enhanced internal edge removal", instead adds runtime checks to detect whether an edge is active or inactive, based on the contact points of the two bodies. This has the benefit of applying not only to collisions with ConcavePolygonShape3D and HeightMapShape3D, but also edges between any shapes within the same body.

Enhanced internal edge removal can be toggled on and off for the various contexts to which it's applied, using the physics/jolt_physics_3d/*/use_enhanced_internal_edge_removal project settings.

Memory usage

[Click to expand/collapse]

Jolt uses a stack allocator for temporary allocations within its simulation step. This stack allocator requires allocating a set amount of memory up front, which can be configured using the physics/jolt_physics_3d/limits/temporary_memory_buffer_size project setting.

Jolt also makes use of aligned allocations for some of its memory usage, which it acquires through function pointers that the implementer assigns. Aligned allocations were recently added to Godot in the form of Memory::alloc_aligned_static, Memory::realloc_aligned_static and Memory::free_aligned_static, which have all been hooked up to Jolt. However, these aligned allocation functions don't currently touch Memory::mem_usage, which means that some of Jolt's memory won't be tracked, and thus the performance monitors in Godot won't be accurate.

Ray-cast face index

[Click to expand/collapse]

The face_index property returned in the results of intersect_ray and RayCast3D will by default always be -1 with Jolt, with a project setting (physics/jolt_physics_3d/queries/enable_ray_cast_face_index) to enable them. The reason for this being that Jolt does not store these indices, and for them to be stored we need to enable the per-triangle userdata, which adds about 25% extra memory usage to the underlying Jolt implementation of ConcavePolygonShape3D.

This should maybe be made into a ConcavePolygonShape3D property instead, through something like what's been suggested here.

Kinematic RigidBody3D contacts

[Click to expand/collapse]

When using Jolt, a RigidBody3D frozen with FREEZE_MODE_KINEMATIC will by default not report contacts from collisions with other static/kinematic bodies, for performance reasons, even when setting a non-zero max_contacts_reported. If you have many/large kinematic bodies overlapping with complex static geometry, such as ConcavePolygonShape3D or HeightMapShape3D, you can end up wasting a significant amount of CPU performance without realizing it.

For this reason this behavior is opt-in through the project setting physics/jolt_physics_3d/simulation/generate_all_kinematic_contacts.

This should probably be made into an RigidBody3D property instead, through something like what's been suggested here.

Contact impulses

[Click to expand/collapse]

Jolt is not able to provide any impulse as part of its contact data, due to how it orders its simulation step, and instead provides a helper function for estimating what the impulse would be based on various parameters. While this is probably fine for most use cases, like emitting a sound based on how hard something collided, it won't be accurate if a body is colliding with multiple bodies during a simulation step.

Area3D and SoftBody3D

[Click to expand/collapse]

This module does not currently support any interactions between SoftBody3D and Area3D, such as overlap events, or the wind properties found on Area3D. Support for this has been added to Jolt recently, and @jrouwe also did some work towards the extension part of it, but it has not been included in this pull request.

ConvexPolygonShape3D

[Click to expand/collapse]

Godot Physics currently skips calculating a proper center-of-mass and inertia for ConvexPolygonShape3D, and instead always uses the shape's local position as its center of mass, while crudely estimating the inertia. Jolt on the other hand does calculate a more accurate center-of-mass and inertia for them. As a result a non-frozen RigidBody3D with such a shape in it can behave differently when comparing the two engines.

WorldBoundaryShape3D

[Click to expand/collapse]

WorldBoundaryShape3D, which is meant to represent an infinite plane, is implemented a bit differently in Jolt compared to Godot Physics. Both engines have an upper limit for how big the effective size of this plane can be, but this size is much smaller when using Jolt, in order to avoid precision issues.

You can configure this size using the physics/jolt_physics_3d/limits/world_boundary_shape_size project setting.

Axis-locking

[Click to expand/collapse]

The PhysicsBody3D.axis_lock_* properties in Godot Physics are implemented by simply zeroing out the velocities for the selected axes. Jolt instead implements these by calculating a new inverse mass/inertia, similar to how RigidBody3D.lock_rotation works in Godot Physics, which seems to better mitigate energy loss during simulation.

However, Jolt does not allow locking all axes, meaning both linear and angular axes, and an error will be emitted from this module when trying to do so, with the recommendation that you instead freeze the body entirely. While this is a simple enough workaround for RigidBody3D, you cannot currently freeze a PhysicalBone3D, so if you want to lock all axes of a PhysicalBone3D you're forced to resort to calling PhysicsServer3D.body_set_mode yourself.

Notable differences to Godot Jolt

While this module is largely a straight port of the Godot Jolt extension, with a lot of cosmetic changes, there are a few things that are different.

Project settings

[Click to expand/collapse]

All project settings have been moved from the physics/jolt_3d category to physics/jolt_physics_3d.

On top of that, there's been some renaming and refactoring of the individual project settings as well. These include:

• sleep/enabled is now simulation/allow_sleep.
• sleep/velocity_threshold is now simulation/sleep_velocity_threshold.
• sleep/time_threshold is now simulation/sleep_time_threshold.
• collisions/use_shape_margins is now collisions/collision_margin_fraction, where a value of 0 is equivalent to disabling it.
• collisions/use_enhanced_internal_edge_removal is now simulation/use_enhanced_internal_edge_removal.
• collisions/areas_detect_static_bodies is now simulation/areas_detect_static_bodies.
• collisions/report_all_kinematic_contacts is now simulation/generate_all_kinematic_contacts.
• collisions/soft_body_point_margin is now simulation/soft_body_point_radius.
• collisions/body_pair_cache_enabled is now simulation/body_pair_contact_cache_enabled.
• collisions/body_pair_cache_distance_threshold is now simulation/body_pair_contact_cache_distance_threshold.
• collisions/body_pair_cache_angle_threshold is now simulation/body_pair_contact_cache_angle_threshold.
• continuous_cd/movement_threshold is now simulation/continuous_cd_movement_threshold, but expressed as a fraction instead of a percentage.
• continuous_cd/max_penetration is now simulation/continuous_cd_max_penetration, but expressed as a fraction instead of a percentage.
• kinematics/use_enhanced_internal_edge_removal is now motion_queries/use_enhanced_internal_edge_removal.
• kinematics/recovery_iterations is now motion_queries/recovery_iterations, but expressed as a fraction instead of a percentage.
• kinematics/recovery_amount is now motion_queries/recovery_amount.
• queries/use_legacy_ray_casting has been removed.
• solver/position_iterations is now simulation/position_steps.
• solver/velocity_iterations is now simulation/velocity_steps.
• solver/position_correction is now simulation/baumgarte_stabilization_factor, but expressed as a fraction instead of a percentage.
• solver/active_edge_threshold is now collisions/active_edge_threshold.
• solver/bounce_velocity_threshold is now simulation/bounce_velocity_threshold.
• solver/contact_speculative_distance is now simulation/speculative_contact_distance.
• solver/contact_allowed_penetration is now simulation/penetration_slop.
• limits/max_angular_velocity is now stored as radians instead.
• limits/max_temporary_memory is now limits/temporary_memory_buffer_size.

There might be some discussion to be had with regards to migrating the settings values for projects who have previously been relying on the extension.

Joint nodes

[Click to expand/collapse]

The joint nodes that are exposed in the Godot Jolt extension (JoltPinJoint3D, JoltHingeJoint3D, JoltSliderJoint3D, JoltConeTwistJoint3D and JoltGeneric6DOFJoint) have not been included with this module.

Instead of exposing bespoke joint nodes, the existing joint node interfaces should be modified by the physics server, through something like what's been suggested here.

Thread-safety

[Click to expand/collapse]

Unlike the Godot Jolt extension, this module does have experimental thread-safety, including support for the physics/3d/run_on_separate_thread project setting. This is achieved by utilizing the same wrapper server that's used by Godot Physics, called PhysicsServer3DWrapMT, as well as introducing a mutex around the rebuilding of shapes, since concurrent shape-casts could otherwise trigger a race condition.

This has however not been tested very thoroughly, so should be considered experimental.

Query performance

[Click to expand/collapse]

The Godot Jolt extension utilizes a custom container (typically referred to as an "inline vector" or "small vector") in order to avoid heap allocations for physics queries that return more than a single hit, meaning intersect_point, intersect_shape, collide_shape, get_rest_info, cast_motion, body_test_motion, test_move, move_and_collide and move_and_slide.

For the sake of simplifying this port I chose to just use JPH::Array for the query collectors instead, which means that these queries will now always allocate on the heap, likely making them noticeably slower.

It should be trivial to make some bespoke container for the query collectors that behaves like an inline/small vector.

Debug renderer

[Click to expand/collapse]

Jolt provides an interface called JPH::DebugRenderer for rendering its view of the physics simulation. In the Godot Jolt extension I expose this as a custom GeometryInstance3D node called JoltDebugGeometry3D, which proved to be very useful during development, but I figured this module should probably take a different approach, so I have omitted that code.

Debug snapshots

[Click to expand/collapse]

The Godot Jolt extension has the ability to generate what Jolt refers to as "snapshots", where it serializes the state of the physics simulation to a file. These can then be loaded in Jolt's own Samples application, to debug issues more closely there, without needing to deal with Godot itself, which has proved to be quite useful when reporting issues upstream to Jolt.

The code for this is technically included with this module, and can be found as JoltPhysicsServer3D::dump_debug_snapshots, but it wasn't clear to me how to best expose this, so I've left it unexposed for now.

Things left to do

Just to give people an idea about the state of this module, and perhaps encourage future contributions if/when this pull request is merged, here is a rough copy of my to-do list:

Must-have

• Add the ability for physics servers to add/remove/change node properties.
• Using the ability to modify node properties, change the joint interfaces to better match Jolt.
• Using the ability to modify node properties, move some of the project settings to instead be per-body/per-shape properties.
• Add support for interactions between Area3D and SoftBody3D.
• Implement the deferred compound shape mentioned above.
• Add memory usage tracking for aligned allocations.
• Add support for freezing PhysicalBone3D.
• Investigate issues with motion queries mentioned above.
• Resolve frustrations with invalid scaling.

Nice-to-have

• Add migration for the project settings.
• Get rid of heap allocations in shape queries when requested hits are less or equal to default.
• Expose Jolt's debug rendering somehow (maybe as a viewport mode?).
• Expose Jolt's debug snapshots somehow, if only just for dev_build.
• Resolve discrepancy with single-body joints mentioned above.
• Get rid of the body accessors in jolt_body_accessor_3d.h, in favor of just storing the JPH::Body* in JoltObject3D.
• Try remove the use of JPH::PhysicsSystem::GetBodies, to avoid the overhead of iterating over static/sleeping bodies.
• Maybe consider adding a temporary memory allocator that doesn't pre-allocate memory.
• Link Jolt's Jolt.natvis file in MSVC builds for easier debugging.

What about the extension?

If/when this pull request is merged, the Godot Jolt extension will officially be considered to be in maintenance mode going forward, with only bug fixes backported to it, but so long as this engine module doesn't have full feature parity with the extension there will be new releases of the extension to ensure that projects relying on its additional features can continue to function.

If/when this engine module reaches full feature parity with the extension then the extension will be discontinued and its GitHub repository archived.

Attribution

In the interest of not cluttering every file in this new module with the copyright notice of Godot Jolt, as required by its MIT license, consider this my permission to omit it entirely. I have obtained permission from the applicable copyright holders (@jrouwe) to do so as well.

---

(This port is based on godot-jolt/godot-jolt@8f1212e, meaning the latest commit of Godot Jolt as of writing this.)

[jrouwe]

Looking good to me! The diff between godot-jolt and this module is much bigger than I anticipated, must have been a lot of work!

[Ughuuu]

Will the addon discontinue and only this will continue?
Have the tests for godot physics (can't remember the repo name) and/or https://github.com/fabriceci/Godot-Physics-Tests been ran on this to see if they match the results godot physics has? I think I ran them sometimes in an older version and there were some problems, but haven't ran lately). Just so there is a clear way to compare the two and see in the future if there are issues, what they are.

[mihe]

The diff between godot-jolt and this module is much bigger than I anticipated, must have been a lot of work!

Yeah, I thought I had a somewhat Godot-like codebase already, but getting rid of all my not-so-Godot idiosyncrasies was a lot of work indeed. I'm sure there's a few left that I've missed.

The functionality from this PR: #96740 seems to be broken. Looks like it's not filtering out the RID for the selected node anymore and repositioning it over itself.

Thanks, I'll take a look!

Will the addon discontinue and only this will continue?

Assuming this module is actually merged and reaches some kind of feature parity with the extension, yes. I don't see the need to maintain both at that point.

Have the tests for godot physics (can't remember the repo name) and/or https://github.com/fabriceci/Godot-Physics-Tests been ran on this to see if they match the results godot physics has? I think I ran them sometimes in an older version and there were some problems, but haven't ran lately). Just so there is a clear way to compare the two and see in the future if there are issues, what they are.

I'll admit I had forgotten that the physics tests in the demo projects repository even existed, but yeah, they seem fine. Although the performance tests seem to crash with either engine for some reason.

I did run the tests in fabriceci/Godot-Physics-Tests beforehand though, and they seem to report the same failures that I've seen before:

1. CollisionShape3D | testing precision moving [CONVEX 2050V] on static [SPHERE] mouvement [LEFT_TO_RIGHT] > Testing stability

The collision normal generated in this test is indeed slightly off when using Jolt, but I'm not sure it's worth getting hung up on, especially considering how dense the convex polygon is. The test is also done using body_test_motion, so it's possible that this is caused/worsened by one of the several discrepancies there as well, but those won't be resolved in this pull request either way.

2. RigidBody3D | testing Continuous Collision Detection (CCD) > Rigid moving in x with CCD detects collision
3. RigidBody3D | testing Continuous Collision Detection (CCD) > Rigid moving in y with CCD detects collision
4. RigidBody3D | testing Continuous Collision Detection (CCD) > Rigid moving in x with CCD detects collision
5. RigidBody3D | testing Continuous Collision Detection (CCD) > Rigid moving in y with CCD detects collision

These fail because the impulse is applied before the body has been added to the physics space, meaning there's no underlying Jolt body to actually apply the impulse to, which I emit an error for currently. You can make these tests pass if you just swap these two lines around.

The problem is you need to calculate the mass properties in order to apply impulses correctly, which can be expensive. In fact, Godot Physics defers this calculation as well, but still lets you apply impulses, resulting in incorrect behavior, as seen in #75934.

6. PhysicsBody3D | testing the collision between a CharacterBody sphere and a big static cylinder > move_and_collide for sphere and cylinder are detected correctly

This fails because the position error exceeds the tolerance by ~0.006 units. I'm not sure it's worth getting hung up on. It's possible that this is caused/worsened by one of the several discrepancies in body_test_motion as well.

7. RigidBody3D | testing the Sphere stack stability > The bodies are sleeping

Stacked spheres do seem to oscillate more in Jolt, even when setting the Baumgarte stabilization to 1.0. It might be that there's some other setting that can help alleviate this, but this doesn't strike me as a scenario you'll see in a lot of games either way. Stacked boxes seem to behave about as well as with Godot Physics.

[smix8]

Great job mihe, jrouwe, and everyone else involved.

I build this and master and spend some time testing both by running the jolt module and godot physics in parallel on the same scenes and projects. So far I am pleasantly surprised that it already works so well considering that so much code had to be changed for the module.

I did not encounter anything that I would really worry about, no error spam or really broken behavior issues swapping the physics in any of my test projects. In fact a few things that I knew were a little brittle and easy to make explode felt a lot more robust with the jolt module. Yes I encountered a few random issues but ad hoc I am not sure if they are even caused by the physics or just some weird editor or script code. Need to investigate and test some more.

Having the two physics run next to each other it was difficult to spot an actual difference in most situations. The most noticeable for me was the joint behavior. For some reason the joints in the "official" Godot physics tests demo project loved to spin in circles when pushed or moved with the jolt module. Do you know what would cause such a difference in behavior compared to the godot physics and if anything can be done about it?

Here is the link for the demo git (or asset lib was already posted).
https://github.com/godotengine/godot-demo-projects/tree/master/3d/physics_tests

The "official" Godot physics test demo showed some other issues. It actually crashes in the performance test of that demo as the jolt body count of 10240 gets exhausted by that test. Instead of just an error it actually crashes the entire engine in the JoltBody3D::_update_group_filter() function. If that error can not be avoided it should at least happen more gracefully without an engine crash.

I would recommend trying out all those tests in that demo project so the most "obvious" things can be ironed out before this gets into more hands. I am sure there will be some other things only discovered by more broad user testing as is always the case. I think that will be unavoidable by such a huge module change and why there will be both physics for some time to fix and adapt.

Expose Jolt's debug rendering somehow (maybe as a viewport mode?).

If you forward the rendering scenario RID of the viewport to the jolt module for debug rendering it can add whatever it wants with the RenderingServer API without actually "hacking" itself into the Viewport code. Users could even render the entire debug in a different debug viewport this way by just switching viewport scenarios or split things up. The current physics partly hacks its collision contacts inside Viewport nodes which is nothing but ugly. Same with the node based debug shape rendering that omits all server created objects and does not run in sync with the actual physics step and state. So those likely will need settings to not have both debugs flicker over each other.

The collision normal generated in this test is indeed slightly off when using Jolt, but I'm not sure it's worth getting hung up on

Yes there are some tiny differences in the return values between the two physics or in general but seriously we are talking about an "error" margin that I find totally acceptable and negatable in context of game engine physics. I know some people can be really riled up about such details but a friendly reminder this is a game engine and not a scientific simulation, the physics need to work in game engine context. It is not about simulating stuff so we can shoot and bounce stuff with pinpoint accuracy to the moon, it is about making games work without the physics exploding left, right and center at every shape seam or step while also not running at 1 fps ... or at least that should imo be the "humble" goal before we worry about those other things.

[mihe]

Yes I encountered a few random issues but ad hoc I am not sure if they are even caused by the physics or just some weird editor or script code. Need to investigate and test some more.

Please do try to share repros of whatever issues you run into.

For some reason the joints in the "official" Godot physics tests demo project loved to spin in circles when pushed or moved with the jolt module. Do you know what would cause such a difference in behavior compared to the godot physics and if anything can be done about it?

I would need to see some kind of repro of this. I haven't had any reports about this for the extension that I can remember, nor have I seen any regressions for joints when using this module.

EDIT: I misread. I'll take a look.

The "official" Godot physics test demo showed some other issues. It actually crashes in the performance test of that demo as the jolt body count of 10240 gets exhausted by that test. Instead of just an error it actually crashes the entire engine in the JoltBody3D::_update_group_filter() function. If that error can not be avoided it should at least happen more gracefully without an engine crash.

I had glossed over this due to Godot Physics also crashing for me in the broadphase performance test (which seems to be from a stack overflow) so I figured it was something with the test itself, but it seems that the crash with this module was indeed a sloppy regression on my part. That's been fixed now, and you should instead get a "graceful" outcome in the form of error spam about needing to increase the body limit.

[mihe]

The functionality from this PR: #96740 seems to be broken. Looks like it's not filtering out the RID for the selected node anymore and repositioning it over itself.

@ryevdokimov You should find that this works now.

[Zylann]

To mitigate the cost of this potentially expensive rebuild, I made it so that shape changes are only ever "committed" to Jolt when the body has entered into a scene tree

Is this still true regarding shape creation as well?
In my voxel terrain, I regularly create a lot of ConcavePolygonShape3D at runtime for each chunk, but it's expensive because it involves BVH calculations. The inability to do this from inside my own meshing threads has limited streaming performance for a long time, since I had to defer it to the main thread. I did this originally because previous physics engines in Godot had no guarantee that this was safe. Now even if this is made safe, I need a way to have control over when the heavy lifting happens (see also godotengine/godot-proposals#483, though a bit old).

[ryevdokimov]

@ryevdokimov You should find that this works now.

Looks good, I'll mark the comment I made as resolved.

[akien-mga]

I've reviewed the thirdparty code integration, and skimmed through some of the module code, this is very solid work.

The code style seems pretty good, I could find some nitpicks but not really worth pointing out on such a massive code conversion from extension to module.

Tested briefly with the GDQuest 3D TPS demo, it seems to work well as a drop-in replacement.

I would suggest making the project setting for the physics engine a "basic" setting so it's easy to find for users who want to test Jolt. And for good measure the 2D one can likely be too, since there are some GDExtensions available for Box2D and Rapier2D I believe.

[mihe]

I would suggest making the project setting for the physics engine a "basic" setting so it's easy to find for users who want to test Jolt.

Would I do that in this PR?

---

Also, as mentioned in the pull request description, and mentioned in passing in other conversations, this module should ideally be considered experimental for the near future. With that in mind, do we perhaps want to change the registered server name to be something like JoltPhysics (Experimental) rather than JoltPhysics, so it's very obvious, or is it fine to just have it be mentioned somewhere in the release blog post?

(I guess that also begs the question why there isn't a space inbetween Jolt and Physics, if we want to bikeshed even more.)

[akien-mga]

I would suggest making the project setting for the physics engine a "basic" setting so it's easy to find for users who want to test Jolt.

Would I do that in this PR?

Yeah I would suggest so, so it's easy to test right away. And it's a pretty minor change well justified by adding a secondary builtin physics engine for 3D.

Also, as mentioned in the pull request description, and mentioned in passing in other conversations, this module should ideally be considered experimental for the near future. With that in mind, do we perhaps want to change the registered server name to be something like JoltPhysics (Experimental) rather than JoltPhysics, so it's very obvious, or is it fine to just have it be mentioned somewhere in the release blog post?

We'll mention it in the blog post for sure, but if you want to make the visible name include (Experimental), I think that's fine. I suppose the actual integer value won't change so when we decide to drop the (Experimental) bit it won't break compat.

(I guess that also begs the question why there isn't a space inbetween Jolt and Physics, if we want to bikeshed even more.)

It can definitely be Jolt Physics (Experimental) :D

The name there should be the canonical name of the library. I think for Bullet Physics we just had Bullet, but in retrospect it should likely have been Bullet Physics if it's how the project calls itself. Likewise for Jolt Physics.

GodotPhysics3D just happens to be the canonical name we chose for Godot's internal unnamed physics backend, and likewise GodotPhysics2D for 2D. It's not a requirement that physics backends should be PascalCase or with a dimensional suffix :)

[mihe]

I suppose the actual integer value won't change so when we decide to drop the (Experimental) bit it won't break compat.

There is no integer value backing this, unfortunately. The physics/3d/physics_engine project setting is a string, which is the registered server name. So this would presumably need some sort of migration code when we decide to drop the (Experimental) part.

Should I still go ahead with adding it?

[akien-mga]

Then I wouldn't add the (Experimental) part, we'll just make it clear elsewhere.

[mihe]

If Jolt is going to be the default after going out of "experimental" then I believe changing the name wouldn't matter since it'd just fall back on itself.

It would still keep the Jolt Physics (Experimental) value for the setting itself though, which will manifest as a blank value in the project settings dialog, even when re-saving the project.

Frankly, the silent fallback that's currently happening should be an error/warning instead.

---

Anyway, so...

1. ⚠️ The registered server name is now Jolt Physics instead of JoltPhysics, so anyone who has messed around with this PR will need to change physics/3d/physics_engine once again.
2. Both physics/3d/physics_engine and physics/2d/physics_engine are now considered basic settings, and thus won't require toggling "Advanced Settings" in the project settings dialog anymore.

I don't believe there are any more pending issues to resolve.

[akien-mga]

Thank you @mihe and @jrouwe for this amazing contribution!

Jolt has proven to be an excellent fit for Godot, especially after all the work done upstream to accommodate our high level API and ensure we can preserve as much compatibility as possible.

The result as of this PR is a drop-in replacement that covers most of the API and should work well for most games. The remaining parts which don't have parity yet will be evaluated in coming months while our Godot Jolt integration goes from experimental to production ready, and eventually if all goes well the new default physics engine.

Thanks also for all the work put into godot-jolt, which helped ensure and proved that GDExtension is suitable for adding major functionality like a whole new physics API, and to iterate quickly in a community project without being held back with the upstream core bureaucracy. Being able to then convert the GDExtension using godot-cpp to a core module was also part of the intended design of godot-cpp, aiming to keep as much of the API matching what Godot C++ modules can do.

Let's get this well tested and ironed out for the 4.4 release!

[mihe]

Awesome! I'm excited to see where things go from here. Hopefully we can smooth out whatever rough edges there are currently and eventually make Jolt the default 3D physics engine.

Also, a big thank you to everyone who took the time to try out the extension and report issues during these past two years. This module would likely have been in a much rougher state (if at all) without the battle-testing from the community.

[jrouwe]

This is very cool! My thanks goes out to @mihe for spending so much time in making this plugin. Jolt really improved due to all the issues he found and requests he made! It has been a joy working together!

Looking forward to the last push to make it production ready! Feel free to tag me on any issues that are reported. Also it would be nice if there was a label that I can filter on to quickly find issues reported against this module, e.g. label:topic:joltphysics.

[akien-mga]

Also it would be nice if there was a label that I can filter on to quickly find issues reported against this module, e.g. label:topic:joltphysics.

We could add this easily, but it would be quite specific compared to our other labels which typically encompass whole systems (like topic:physics).

We do have a need for more granularity but labels won't cut it (we'd need to multiply the amount of labels by 3-5 to cover everything), instead we're working on preparing some public GitHub projects for each team/system (basically each topic: label), which will have further categorization. So once this is done you should be able to find and filter issues with the Jolt Physics category in the (upcoming) Physics team issue triage project.

[albinaask]

First of all. Nice work.

One thing that would be a game changer for my game project, which heavily relies on modifying compound, rigid bodies with sometimes hundreds of shapes would be to be able to offload the updating process of these bodies to a worker thread. This would cause the body, and then replacing the body a few frames later. This would greatly reduce the risk of frame rate drops upon adding or removing shapes to a body. The trade-off to wait for a frame or two to get the new shape in place would in this case be more than reasonable in my opinion. This could practically be achieved through adding an opt-in bool parameter in the PhysicsServer3D.body_[add/remove]_shape or by a project setting. Would you @mihe think this would be an impossibility?

On another note. Will Godot's 64-bit precision option work within this framework? How will Godot integrate Jolt's ability to work with vector instructions (SSE, AVX etc)?

[Zylann]

On another note. Will Godot's 64-bit precision option work within this framework?

It should:

godot/modules/jolt_physics/SCsub

Line 155 in b9437c3

if env["precision"] == "double":

See also about how it was added: jrouwe/JoltPhysics#94 (comment)

[yythlj]

Is the common/physics_interpolation function of Godot adapted to JOLT?@mihe
（https://github.com/godotengine/godot/pull/92391）

[Calinou]

Is the common/physics_interpolation function of Godot adapted to JOLT?@mihe （godotengine/godot/pull/92391）

Yes, physics interpolation works out of the box with Jolt. It's not performed in the physics engine, but within the rest of the engine, so it works with third-party physics engines too. It'll even work if you write your own physics in _physics_process() assuming the node has physics interpolation enabled.

[wagnerfs]

I suppose jolt being now part of the Godot engine, I should file bug reports for it in Godot Engine's issues, right?

[akien-mga]

I suppose jolt being now part of the Godot engine, I should file bug reports for it in Godot Engine's issues, right?

To be clear, the Jolt project is still a standalone library that has its own development priorities and issue tracker.

If you have issues with Jolt in Godot, then yes those issues should be filed in the Godot repository, so that we can assess whether the problem comes from the Godot integration, or is a library issue that needs to bubble up upstream.

[mihe]

If you have issues with Jolt in Godot, then yes those issues should be filed in the Godot repository

To add to this, if the bug is manifesting with the Godot Jolt extension then you're welcome to continue reporting them to the godot-jolt/godot-jolt repository, just to avoid burdening the Godot repository with stuff that might perhaps be extension-specific. I'll fix whatever needs fixing in both repositories for the foreseeable future, and cross-report the issue if it's something more long-term.

If the bug is manifesting in this new engine module, then definitely report it to the Godot repository.

One thing that would be a game changer for my game project, which heavily relies on modifying compound, rigid bodies with sometimes hundreds of shapes would be to be able to offload the updating process of these bodies to a worker thread. This would cause the body, and then replacing the body a few frames later. This would greatly reduce the risk of frame rate drops upon adding or removing shapes to a body. The trade-off to wait for a frame or two to get the new shape in place would in this case be more than reasonable in my opinion. This could practically be achieved through adding an opt-in bool parameter in the PhysicsServer3D.body_[add/remove]_shape or by a project setting. Would you @mihe think this would be an impossibility?

@albinaask This discussions is probably better suited for a proposal, but it's certainly not impossible. The question is more of whether there are other approaches that can help alleviate this without burdening the shared PhysicsServer3D interface or throwing threads (and potential concurrency issues) at the problem, like introducing the hybrid/deferred compound shape mentioned in the PR description, and/or allowing for the use of Jolt's MutableCompoundShape instead.

How will Godot integrate Jolt's ability to work with vector instructions (SSE, AVX etc)?

@albinaask As far as I know, Jolt should automatically make use of whatever instruction sets are enabled/defined during compilation, which for Godot is SSE2 by default. If you want more than this (SSE4.2, AVX, etc.) you'll need to pass the appropriate compiler options (e.g. -mavx for GCC/Clang) through something like Godot's ccflags SCons option.