Pick instance feature #479

thatcomputerguy0101 · 2024-09-11T20:25:38Z

This adds pick_instance and ray_intersect_instance functions to allow detecting which geometry instance a ray intersects with. There is a new example picking_instances that demonstrates this capability for basic geometry, instanced geometry, and imported models. The additional example has yet to be added to the examples README. This closes #467.

Additionally, the shader IDs have been separated out to simplify keeping track of which ID's were used, and a few ID overlaps were corrected. The shader IDs internally make use of open_enum types, which could be changed to also be the public type signature in a breaking release. The shader ID changes can be seen individually at https://github.com/thatcomputerguy0101/three-d/tree/shader_ID_refactor, and I can create a separate pull request for that or remove those changes if desired.

asny · 2024-09-17T11:14:50Z

@thatcomputerguy0101 I made a separate PR for the shader ID refactor to make it easier to review and test that change separately. I'll update this PR after that is merged.

thatcomputerguy0101 · 2024-09-17T14:11:15Z

This PR has now been updated to be based on the current master branch.

asny · 2024-09-20T09:26:05Z

examples/picking_instances/src/main.rs

Super nice with an example, but maybe we can just extend the current picking example to also change the color of the selected mesh/instance.

asny · 2024-09-20T09:26:26Z

examples/picking_instances/src/main.rs

+    let mut sphere_instances = Instances {
+        transformations: [
+            Mat4::from_translation(vec3(2.0, 0.0, 0.0)),
+            Mat4::from_translation(vec3(4.0, 0.0, 0.0)),


Would be nice with a lot of instances.

I am confused if this is a suggestion for a change or just a comment.

It is a suggestion, currently, there's only two instances, it's better to test with 100 instances for example.

asny · 2024-09-20T09:28:27Z

src/renderer.rs

+///
+/// Representation of a specific geometry instance
+///
+pub struct GeometryInstance {


I'm not sure I like the name (naming is hard). It's not a geometry instance, it's not even necessarily an picking result containing an instance. Maybe PickResult or something?

Yeah, it used to fit the GeometryInstance name better, but has since been adjusted away from that. However, ray_intersect_instance also returns that type, so maybe IntersectionResult would be good?

asny · 2024-09-20T09:41:00Z

src/renderer.rs

+/// and (viewport.x + viewport.width, viewport.y + viewport.height) indicate the top right corner.
+/// Returns ```None``` if no geometry was hit between the near (`z_near`) and far (`z_far`) plane for this camera.
+///
+pub fn pick_instance(


These functions are almost identical to pick and ray_cast, maybe just update those return more data. It's a breaking change, but an easy one to understand and fix. Also, it shouldn't affect performance, so even if you only want the picked position, the new functionality should still be good.

I'll probably change that with my next set of edits. (Technically it is a couple of instructions slower due to moving more data around, but that is probably a negligible difference especially once compiler optimizations are applied.)

asny · 2024-09-20T09:41:55Z

src/renderer.rs

+        texture.as_color_target(None),
+        depth_texture.as_depth_target(),
+    )
+    .clear(ClearState::color_and_depth(-1.0, -1.0, -1.0, -1.0, 1.0))


The clear values are clamped between 0 and 1, so this doesn't make sense.

The platform that I was testing on doesn't seem to be clamping it, which was why I thought it was fine. I'll try and find an alternative solution to have the intended behavior on all platforms.

Hmm. That's weird, but yeah, you never know how OpenGL specs are actually implemented. I guess you can use 0 for no hit and just offset the ID result with -1.

asny · 2024-09-20T09:53:10Z

src/renderer/material/geometry_instance_material.rs

+#[derive(Default, Clone)]
+pub struct GeometryInstanceMaterial {
+    /// Geometry ID
+    pub id: usize,


I generally try to use u32 or u16 to make it explicit. usize varies with operating system and is usually u64 which can't fit in a f32 anyway.

src/renderer/material/geometry_instance_material.rs

src/renderer/material.rs

src/renderer/geometry/shaders/mesh.vert

src/renderer/material/geometry_instance_material.rs

thatcomputerguy0101 · 2024-09-24T02:19:48Z

I'm looking at using ints the whole way through the GPU side, although that requires using an integer output texture which restricts support to OpenGL 3.0 or greater and WebGL 2. Would these be acceptable versions to still replace the original pick, or should pick_instance remain separate to support older versions that only need pick?

asny · 2024-09-25T12:09:58Z

I'm looking at using ints the whole way through the GPU side, although that requires using an integer output texture which restricts support to OpenGL 3.0 or greater and WebGL 2. Would these be acceptable versions to still replace the original pick, or should pick_instance remain separate to support older versions that only need pick?

Yes, that would be ok. Would you write the depth to an integer then? You need to write depth (f32) and ID (u32) to a render target, so you have to choose either a float or integer target. I'm not sure it makes a big difference whether you choose one over the other.

thatcomputerguy0101 · 2024-09-25T17:20:33Z

Part of that change would be reading the depth from the depth buffer (supported in WebGL 2 if I interpreted it correctly), which is always a float data type.

thatcomputerguy0101 · 2024-09-28T14:32:12Z

Seems like I misinterpreted being able to read from the depth texture on web. I still have some ideas to work around that though.

thatcomputerguy0101 · 2024-09-28T18:28:16Z

GL's floatBitsToInt function ended up being useful. It is fully working on both MacOS and web with an integer output texture, and the floating point depth value encoded in one of the color channels. I'll probably create a separate PR for integer texture support, as I made some significant changes to get that working.

asny · 2024-10-04T16:04:05Z

@thatcomputerguy0101 These changes seems a bit too much for the feature, so I tried implementing it myself (picking snippets from this PR and #478). You can see the result in #494, it's significantly simpler and works well. So is there a good reason for all of these changes?

asny mentioned this pull request Sep 17, 2024

Shader ID refactor #484

Merged

thatcomputerguy0101 force-pushed the pick_instance_feature branch from f9c557d to a302f37 Compare September 17, 2024 14:10

asny reviewed Sep 20, 2024

View reviewed changes

thatcomputerguy0101 force-pushed the pick_instance_feature branch 2 times, most recently from 867cc4b to d5cf6c2 Compare September 27, 2024 05:31

thatcomputerguy0101 added 2 commits September 30, 2024 20:03

Add support for integer render targets

ee83d80

Add integer texture support, for creating render targets

973bce5

thatcomputerguy0101 force-pushed the pick_instance_feature branch from d5cf6c2 to 5d4ce11 Compare October 1, 2024 01:46

thatcomputerguy0101 mentioned this pull request Oct 1, 2024

Integer render target #490

Closed

thatcomputerguy0101 added 6 commits October 1, 2024 00:00

fmt

c9d0a98

Add pick geometry instance

1ec8791

Add pick geometry instance example

0cd65c0

Add web console logs to pick_instance example

0761bbc

Pass IDs in to shader as integers

288c899

Read from ID shader in integer format

1d650d8

thatcomputerguy0101 force-pushed the pick_instance_feature branch from 5d4ce11 to 1d650d8 Compare October 1, 2024 04:01

asny mentioned this pull request Oct 4, 2024

Improved picking #494

Merged

thatcomputerguy0101 closed this Oct 5, 2024

thatcomputerguy0101 deleted the pick_instance_feature branch October 5, 2024 17:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pick instance feature #479

Pick instance feature #479

thatcomputerguy0101 commented Sep 11, 2024

asny commented Sep 17, 2024

thatcomputerguy0101 commented Sep 17, 2024

asny Sep 20, 2024

asny Sep 20, 2024

thatcomputerguy0101 Sep 20, 2024

asny Sep 25, 2024

asny Sep 20, 2024

thatcomputerguy0101 Sep 20, 2024 •

edited

Loading

asny Sep 20, 2024

thatcomputerguy0101 Sep 20, 2024

asny Sep 20, 2024

thatcomputerguy0101 Sep 20, 2024

asny Sep 25, 2024

asny Sep 20, 2024

thatcomputerguy0101 commented Sep 24, 2024

asny commented Sep 25, 2024

thatcomputerguy0101 commented Sep 25, 2024

thatcomputerguy0101 commented Sep 28, 2024

thatcomputerguy0101 commented Sep 28, 2024

asny commented Oct 4, 2024

Pick instance feature #479

Pick instance feature #479

Conversation

thatcomputerguy0101 commented Sep 11, 2024

asny commented Sep 17, 2024

thatcomputerguy0101 commented Sep 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thatcomputerguy0101 Sep 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thatcomputerguy0101 commented Sep 24, 2024

asny commented Sep 25, 2024

thatcomputerguy0101 commented Sep 25, 2024

thatcomputerguy0101 commented Sep 28, 2024

thatcomputerguy0101 commented Sep 28, 2024

asny commented Oct 4, 2024

thatcomputerguy0101 Sep 20, 2024 •

edited

Loading