Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for developer-created spatial anchors via XR_FB_spatial_entity #121

Merged
merged 1 commit into from
May 7, 2024

Conversation

dsnopek
Copy link
Collaborator

@dsnopek dsnopek commented Apr 15, 2024

This is a follow-up to PR #107 which added support for scene anchors (aka the data from Meta's scene understanding).

This PR adds the ability for the developer to create and manage their own spatial anchors.

Here's a screenshot from the demo:

org godotengine openxr vendors demo-20240415-165709

When you switch into passthrough mode, you can press the trigger on a surface from scene understanding (aka a scene anchor) and place a spatial anchor, which is represented by a colored cone (the color is selected randomly). If you point at an existing spatial anchor, pressing the trigger will remove it. If you close and open the app again, the spatial anchors will be restored with the same colors they had previously.

This is done via a new node called OpenXRFbSpatialAnchorManager which works a lot like the OpenXRFbSceneManager which was added in PR #107

Here's how it works:

  • You can create an anchor by calling create_anchor(transform: Transform3D, custom_data: Dictionary) on the node. It won't be created immediately - it's an asynchronous operation. But when it is, the openxr_fb_spatial_anchor_tracked signal will be emitted, which will give the developer an XRAnchor3D and the OpenXRFbSpatialEntity object (with the custom_data stashed in it). The custom_data can be used to store application-specific data about what the anchor actually is, ie what object the player is actually placing in the world (on the OpenXR side it only has a UUID and a location).
  • I suspect a very common case will be creating an instance of a particular scene as a child of the XRAnchor3D. So, rather than connecting to the openxr_fb_spatial_anchor_tracked signal and instantiating the scene manually, the developer can optionally set the scene property, which will be automatically instantiated, and have a method called on it with the OpenXRFbSpatialEntity. (This is very similar to how OpenXRFbSceneManager works!) Here they can grab the custom_data and use it to initialize the scene - the demo uses this to setup the anchor with the correct color.
  • By using the persist_in_local_file and local_file_path, the OpenXRFbSpatialAnchorManager node will save the anchors to a JSON file (along with their custom_data), and then automatically load them from the file when the application starts up again. While it's possible to query OpenXR for all pre-existing anchors, all you get is the UUIDs and positions, you don't get any information about what the anchor is meant to be in the application. That's why we need to also store these in a JSON file along with the custom_data, which is where the developer would put that information.
  • The openxr_fb_spatial_anchor_tracked signal is also emitted for anchors that are loaded (not just created), which allows the developer to handle new and existing anchors at the same time.
  • If the developer doesn't want any of the automatic file saving/loading or other automatic features, they can disable those via properties on the node, which gives them the freedom to do something more custom if they want.

UPDATE: The saving/loading from file functionality was removed from the node, and moved into the demo project. Developers will need to implement their own loading/saving, but once they hand the data off to the node, it should take care of everything else.

Here's the properties in the editor:

Selection_149

@dsnopek dsnopek added the enhancement New feature or request label Apr 15, 2024
@dsnopek dsnopek added this to the 3.0.0 milestone Apr 15, 2024
@dsnopek dsnopek requested review from m4gr3d and BastiaanOlij April 15, 2024 22:24
@dsnopek dsnopek force-pushed the fb-create-spatial-anchors-api branch 4 times, most recently from d88a214 to d57232c Compare April 22, 2024 15:28
@dsnopek
Copy link
Collaborator Author

dsnopek commented Apr 22, 2024

Rebased after recent merges!

@dsnopek dsnopek force-pushed the fb-create-spatial-anchors-api branch 2 times, most recently from 266c6ed to f9bbbec Compare April 26, 2024 20:28
@dsnopek dsnopek force-pushed the fb-create-spatial-anchors-api branch 2 times, most recently from ca93e51 to ea04f10 Compare April 30, 2024 20:11
@dsnopek dsnopek requested a review from m4gr3d April 30, 2024 20:15
@dsnopek dsnopek force-pushed the fb-create-spatial-anchors-api branch 2 times, most recently from 0b0eb76 to 1503d73 Compare May 2, 2024 16:05
@dsnopek
Copy link
Collaborator Author

dsnopek commented May 2, 2024

This is updated per @m4gr3d's requested changes, and rebased on all the recent merges

nullptr, // next
reinterpret_cast<XrSpace>(get_openxr_api()->get_play_space()), // space
pose, // poseInSpace
get_openxr_api()->get_next_frame_time(), // time
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use get_predicted_display_time following the merging of godotengine/godot#89734

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! However, we'd need to update the extension_api.json we're using, and there's 4 other unrelated places where we're calling get_next_frame_time(). That feels like something for a dedicated PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! I've added #140 to track.

@dsnopek dsnopek force-pushed the fb-create-spatial-anchors-api branch from 1503d73 to 95f1ced Compare May 7, 2024 20:01
Copy link
Collaborator

@m4gr3d m4gr3d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@m4gr3d m4gr3d merged commit 32f7203 into GodotVR:master May 7, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants