IDL proposal: Expose the list of anchors in XRSession.

[judax]

Should a list of anchors be exposed in the XRSession at all times? Is it useful? What implications has (keeping it correctly up to date when anchors are added/removed).

[blairmacintyre]

What would web devs expect? How do other APIs handle this? You can get lists of DOM elements, for example.

Why would you not want a list of current anchors available? This feels like "a thing that will be commonly useful that we should probably just do once".

[blairmacintyre]

Also, if anchors have IDs, this will allow searching.

So, anchors created via hit test and by the programmer as absolute coordinates might have "GUID" names generated for them, but if we want to expose other things (like, say, making an anchor for the origin of the VR stage), we could create a "stage" anchor. Programmers might then search for a stage anchor, and if one isn't found, ask the user to select a spot for their content.

[lincolnfrog]

Theoretically we could create a JS library that handles this since all the information needed to create and update the list is already there in the event-driven API. I would prefer we keep the native APIs as lean as possible and rely on JS libraries where possible when different handles are needed. I have a strong feeling, given the ergonomics of the WebXR spec, that there will be coarser building blocks for creating these experiences anyway for stuff like removing session creation boilerplate, etc.

[blairmacintyre]

I'm fine with that; in the WebXR Viewer, that's exactly the way it works in my samples. Any "existing" anchors that are around when an app begins result in immediate "onCreate" events, so the app gets notified of existing anchors.

[ddorwin]

I think we should avoid doing this unless/until we know that it is a good idea. As was already mentioned, apps and libraries can easily do this, but specing it could limit future API changes and implementations.

It's possible that such a list will be a necessary for other reasons (i.e., #12), but we should consider it as part of those discussions rather than independently.

[judax]

My take on this (and that is why I did not add it to the IDL proposal) is that this list is not necessary. I can't come up with a use case that can't be solved without the list. Could it be a "nice to have" in some situations? Yes. But creating a list of anchors by the developer when needed should not be a blocker. I would encourage to anyone supporting the idea of having the list to describe some use cases that could shed some light. The ones I am thinking right now actually are even more counter productive as they could lead developers to use polling instead of the events designed for it (having the up to date list of anchors might be tempting for some devs).

[blairmacintyre]

I agree; as long as we reliably issue create/destroy events when anchors are created and destroyed, and if there are "pre-existing anchors" that will cause a bunch of create events at the start, we are clear when this will happen so that developers can create their event listeners before this time, there is no need for a list.

[judax]

What do you have in mind for "pre-existing anchors"? I was thinking that no event should be fired at least until the session creation request promise is fulfilled.

[blairmacintyre]

Absolutely. Once you have a session, and will start to get rAF calls, at that point the pre-existing anchors should/would be sent in. In the WebXR Viewer, I'm pretty sure I send the events before or after the first rAF (can't recall): I suspect a programmer might not be "ready" for them until they get a session.

[thetuvix]

Yes, a general list of detected environmental features does not feel necessary, given events. To @ddorwin's point about constraining future API development or UA behavior, ARCore/ARKit apps today often reason in quite small spaces, and so getting "all" of something like meshes or planes is currently fine. However, very soon, apps are likely to have access to data across large areas. We may not want to pre-establish an API that promises to give you "all" of anything, before we're sure quite how we'd scope that.

Note that this will concern the events in the API as well, although there you have a more natural pattern for expressing newly-in-scope or newly-out-of-scope elements as the app or user expands/contracts the domain in which the app is reasoning.

[bialpio]

In the current proposal, we seem to not include any information about anchor removal (tracking loss?) events. Same applies to anchor creation events, although they might not be necessary if we assume that every anchor gets created in response to the application’s request (is it safe to assume so?). If we add those 2 events, then the list is not needed and can be done purely in JavaScript.

… but ...

I’m not sure if we should worry that much about handing out lists of “all” of things, especially when we’re considering anchors. Creating anchors can have performance implications so we should educate API users that less is better (see #23). Additionally, say that we have 1000’s of anchors, and they all get updated per frame [0] - even without the list of all anchors, we’d still need to fire 1000’s of updated events (or one event with 1000’s of anchors, see #12), and the UA would still need to make all the anchor data available to the application by updating the anchor objects. Having a list of all anchors would just mean that we additionally store 1000’s of references to the anchor objects, we’re already paying all the other (potentially big) costs with just the event-based API.

I would prefer we keep the native APIs as lean as possible and rely on JS libraries where possible when different handles are needed.

Other side of the argument is that if every app needs to do exactly the same thing, then it’s boilerplate that we can try to reduce by rethinking the API design.

[0] As a personal data point: from my local experiments, I’ve seen that ARCore’s anchors appear to all get updated every 3 frames (~100ms). This is an implementation detail but we might want to consider it in our discussions.

[blairmacintyre]

@bialpio wrote:

Same applies to anchor creation events, although they might not be necessary if we assume that every anchor gets created in response to the application’s request (is it safe to assume so?).

I think so. In earlier versions of the WebXR Viewer, we had the facility to send creation send creation events, but they only get used for "tracked things". The obvious examples are planes, and faces: things that get created explicitly by the system.

The explainer is very explicit about NOT including those sorts of Anchors. The question is, are we expecting that all "other" uses of Anchors (planes, faces, etc) will be handled separately (like the geometry APIs)? If so, perhaps the notifications happen there? Certainly, the current world geometry explainer handles creations/deletion of planes and meshes.

I think this question would help guide the discussion here. If we are assuming that the Anchor list proposed here is only for anchors created here (at explicit poses, or in response to hit testing), we may do one thing. In that case, the set of anchors may only change in response to programmers creating new ones, which (as you point out) will likely not happen continuously/frequently. So, in that case, exposing an array will not be a massive performance problem, because the array itself will not change frequently. Having that array might be convenient.

If, on the other hand, we assume that all of these other world objects (planes, meshes, tracked images, faces, etc) are going to be subtypes of XRAnchor, we might do another thing. In this case, those arrays may change a lot, and continuously, so the other APIs may want to tailor what they expose to the use case. For example, world knowledge (which might have lots of continuously changing lists) might favor one approach, and face/image tracking (which might have a small list of entities that doesn't change change often, even if the entity data changes continuously) may favor another.

My initial thought is we should not expose a list of "all anchors" but rather consider exposing lists of specific ones in specific APIs. Here, I could see supporting a list of anchors created from poses/hit tests, but don't feel that strongly, since these sorts of Anchors would almost definitely be associated 1:1 with applications specific objects (the stuff attached to the anchors!) so apps would be maintaining references to them already.

[bialpio]

In the current draft, I have decided to expose the set of all currently tracked anchors via XRFrame. In the current approach to the API, the anchor is explicitly something that the application creates that attempts to maintain fixed relationship to the real world (i.e. planes would not inherit from anchors). I'm closing the issue, please reopen or file a new one if you disagree with the current approach.

[thetuvix]

Given this approach of XRAnchor only representing explicit app-created freespace anchors, having a list of tracked anchors seems harmless to me.

XRFrame.trackedAnchors does feel lower value than is implied by the code in the anchors explainer, as there is no per-anchor ID or other data beyond the anchorSpace to tie a given XRAnchor back to any set of app content. The app itself will need to remember for each anchored scene object it creates which anchor that object is attached to... at which point the app could just loop over its own list of those anchors and poll for isLost or such.

Note that the for loop over trackedAnchors at the bottom stops after getting each XRAnchor's pose - it's not clear what the next line of code would be for the app to make productive use of that pose, unless it was already maintaining an equivalent trackedAnchors map from scene node to anchor itself:

let previousFrameAnchors = Set();

function onXRFrame(timestamp, frame) {
  frame.session.requestAnimationFrame(onXRFrame);

  const trackedAnchors = frame.trackedAnchors;

  for(const anchor of previousFrameAnchors) {
    if(!trackedAnchors.has(anchor)) {
      // Handle anchor tracking loss - `anchor` was present
      // in the present frame but is no longer tracked.
    }
  }

  for(const anchor of trackedAnchors) {
    // Query most recent pose of the anchor relative to some reference space:
    const pose = xrFrame.getPose(anchor.anchorSpace, referenceSpace);
  }

  previousFrameAnchors = trackedAnchors;
}

The primary way I could see an app productively using trackedAnchors is if it set its own additional attribute on each XRAnchor to store the list of its root scene objects whose poses need to be updated each frame:

  for(const anchor of trackedAnchors) {
    // Query most recent pose of the anchor relative to some reference space:
    const pose = xrFrame.getPose(anchor.anchorSpace, referenceSpace);

    for(const sceneNode of anchor.attachedSceneNodes) {
      // Adjust the pose of each scene node attached to this anchor.
      sceneNode.setTransform(pose.transform);
  }

This seems like a reasonable pattern, although it would rely on a promise that the UA will return the same XRAnchor instance on subsequent frames when XRFrame.trackedAnchors is enumerated, rather than returning a new equivalent XRAnchor instance that lost that extra data.

I'll file a new issue around specifying that the UA must retain any extra data on an XRAnchor when it's enumerated moving forward.