Need to expose geometry in a way that is generic, but doesn't throw out too many semantics

[blairmacintyre]

A simple solution to world geometry is pick a lowest-common representation, like "a forest of meshes".

But, then ARKit/ARCore planes become meshes, and lose semantics: we no longer know that they are meant to correspond to vertical or horizontal planes. Similarly, faces could be exposed as a mesh that comes and goes, as could moving objects or detected images, etc.

So, a slightly less "wasteful" approach might be to say

• we expose a forest of meshes
• a mesh does not need to satisfy any particular geometric properties, aside from sharing a set of vertices that are used to construct the "mesh".
• each mesh has an origin in world coordinates, and be defined relative to it. That origin could change each frame (e.g., so ARKit/Core planes/faces can be represented this way, as can tracked objects if the browser supports such a thing)
• each mesh can be typed, perhaps just using string names, and have additional information with it beyond the mesh that depends on the type. Apps that know of the type can use it, others will just use the mesh.
  • planes might have a surface normal
  • faces might have an array of blend shapes
  • when other things are detectable, they can be exposed similarly. So, if we exposed "image detection and tracking", the detected image could have a mesh defined for it that corresponds to the image in the real world -- same for objects being detected (the real-world equivalent OR the thing used for detection could be returned)

[johnpallett]

Thanks Blair! There are certainly a few approaches we could take here, and exposing individual planes should not preclude exposing mesh data (either as an alternative to planes, or through metadata on the planes)... at the very least, having planes doesn't limit other API options in SLAM systems that I've seen today.

Can you take a look at the explainer and let us know what you think about the approach?

[blairmacintyre]

ha, I will. That wasn't there when I posted this. At a casual glance, I have "comments", will post later (meetings now)

[johnpallett]

@blairmacintyre I think this discussion topic is now captured now in #6 - can you confirm?

[blairmacintyre]

yes, let's close.