-
-
Notifications
You must be signed in to change notification settings - Fork 21.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for OpenXR hand interaction extension #81533
Add support for OpenXR hand interaction extension #81533
Conversation
3850583
to
9a1f7ce
Compare
metadata->register_io_path("/interaction_profiles/ext/hand_interaction_ext", "Pinch pose", "/user/hand/right", "/user/hand/right/input/pinch_ext/pose", "", OpenXRAction::OPENXR_ACTION_POSE); | ||
metadata->register_io_path("/interaction_profiles/ext/hand_interaction_ext", "Poke pose", "/user/hand/left", "/user/hand/left/input/poke_ext/pose", "", OpenXRAction::OPENXR_ACTION_POSE); | ||
metadata->register_io_path("/interaction_profiles/ext/hand_interaction_ext", "Poke pose", "/user/hand/right", "/user/hand/right/input/poke_ext/pose", "", OpenXRAction::OPENXR_ACTION_POSE); | ||
metadata->register_io_path("/interaction_profiles/ext/hand_interaction_ext", "Palm pose", "/user/hand/left", "/user/hand/left/input/palm_ext/pose", XR_EXT_PALM_POSE_EXTENSION_NAME, OpenXRAction::OPENXR_ACTION_POSE); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't the 'palm pose' require the XR_EXT_palm_pose to be enabled?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, though there was talk at some point that this extension would require palm pose to also be available. I'm really hoping palm pose becomes a mandatory thing at some point, it's pretty useless as an optional extension.
Anyway, we support the palm pose extension so if its enable, these paths become active if the extension is supported by the device.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could wrap this in a check for the palm extension?
metadata->register_io_path("/interaction_profiles/ext/hand_interaction_ext", "Palm pose", "/user/hand/left", "/user/hand/left/input/palm_ext/pose", XR_EXT_PALM_POSE_EXTENSION_NAME, OpenXRAction::OPENXR_ACTION_POSE); | |
if (OpenXRPalmPoseExtension::get_singleton()->is_available()) { | |
metadata->register_io_path("/interaction_profiles/ext/hand_interaction_ext", "Palm pose", "/user/hand/left", "/user/hand/left/input/palm_ext/pose", XR_EXT_PALM_POSE_EXTENSION_NAME, OpenXRAction::OPENXR_ACTION_POSE); | |
} |
profile->add_new_binding(primary, "/user/hand/left/input/pinch_ext/value,/user/hand/right/input/pinch_ext/value"); | ||
profile->add_new_binding(primary_click, "/user/hand/left/input/pinch_ext/ready_ext,/user/hand/right/input/pinch_ext/ready_ext"); | ||
|
||
// Use activation as secondary |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The spec is unclear, what's the role of aim_activate_ext
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean the OpenXR spec?
The "aim_activate" gesture is runtime defined, and it should be chosen so that the "aim" pose tracking is stable and usable for pointing at a distant target while the gesture is being performed.
That is indeed extremely vague, so the runtime defines the situation it is activated by detecting you're pointing at something. How it does that? 🤷
So far I was under the impression this basically returns 1.0
/true
when you extend your finger, but I could be wrong about that.
Personally this is the only element of this new profile I'm unlikely to use, until someone shows me a good use case :)
Just double-checking that you're including the hand tracking permissions when testing this extension? |
a2e76ae
to
4ef64f0
Compare
As far as I'm aware yes, I've combined the logic with the normal hand tracking demo and the hand tracking works, but the interaction doesn't. This could simply be because its gated behind the OpenXR version or because it's not yet enabled on the current Quest OS. Anyway, I've added some basic logic around this here: godotengine/godot-demo-projects#973 |
6a87026
to
d882031
Compare
d882031
to
e18ed03
Compare
Just a bit of extra info on the progress with this PR. Basically the functionality is finished but we're currently waiting for XR Vendors to take the implementation of the feature out of beta. |
e18ed03
to
9007a32
Compare
9007a32
to
7dd0939
Compare
Note regarding CI failure, edit I've undone this change as not to upset GDExtensions etc. It's not directly related to this PR. we can always make a separate PR if this is important. |
Because the hash changes, it would really break compatibility. However, adding compatibility functions for them should be pretty easy! There's a number of examples of them in the code base to look at, for example, Or, if you can't work it out, just let me know and I can make a patch :-) |
Was this in relation to this PR David? I think you might have meant to leave this as a comment on one of Giles PRs? |
Yes, my comment was meant for this PR! I was just pointing to Gilles' changes as an example for how to handle this situation, since he also had a method change to This PR will need to add some compatibility methods because adding/removing |
6408a83
to
f04bde9
Compare
I ended up undoing the change, it's not really related to this PR, just something I noticed for existing calls that we normally make consts. I think that if its important, we can make a separate PR. |
I'm changing this back to ready for review. Right now the only HMD that implements this extension is the Magic Leap 2, but I'm hoping that other headsets follow suit. I think it's worth merging this for 4.3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have the hardware to test this, but overall the code looks good to me!
f04bde9
to
527c30c
Compare
Thanks! |
This PR adds support for OpenXRs hand interaction extension that was introduced in 1.0.28.
This is a long awaited interaction that allows us to use the action system when optical hand tracking is used. This makes it much easier to create a portable application that works with hand tracking and controller tracking.
The extension introduces two new poses:
It also adds 3 gesture based inputs:
When both a hand interaction profile and a controller interaction profile is supplied, the XR runtime will switch between the profiles depending on whether the user is holding a controller or whether optical hand tracking is active.
However if you only supply a hand interaction profile in your action map, the OpenXR specification states that any XR runtime that supports this extension should use the hand interaction profile even if a controller is being held.
This is ideal for projects that are designed toward hand tracking but want a controller fall back, especially in AR/Passthrough applications.
See: https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_EXT_hand_interaction
A WIP demo project with basic functionality can be found here: godotengine/godot-demo-projects#973
Note: This PR is basically done, I need to do some testing and create a nice example project.