Controller info: 3D models and control position metadata

[TrevorFSmith]

How should we go about getting / making info for the hardware that we want to support?

I was thinking that for each type of controller we would ideally have:

glTF models

left and right handed models of a consistent size (1u = 1M)

hardware control position metadata

the center position and plane of each button, pad, and stick.

pointing ray

on tracked hardware, the ray that is most naturally used to point

I think that @cvan mentioned that there have been previous conversations about this. Chris, do you know whether anyone started to gather these models?

[cvan]

short answer: no, but should probably coordinate or ping the authors of the Gamepad Input API proposal: https://docs.google.com/document/d/1nr7AdK3KcS3k5usvUU2W6R_feSlSGVFMYHqqzI2pC0c

@toji is probably a good Point of Contact for connecting with someone who may already be working on this.

I know @bluemarvin just took some from Google Poly (or was it Sketchfab?) for the Daydream OBJ controller models in his demo.

[fernandojsg]

We have almost all the models on https://github.com/aframevr/assets/tree/gh-pages/controllers with open licenses, we could just convert them to gltf ourselves and rename the nodes as needed

[netpro2k]

Maybe rather than (or in addition to) position metadata, information about the nodes in the gltf for the corresponding inputs should also be provided, for example the node id, or a path to the node. Providing a reference itself wouldn't really work as loading the gltf is going to be framework specific.

[fernandojsg]

@netpro2k yep! my idea is to follow the original webvr proposal (https://github.com/immersive-web/webxr/pull/279/files) using the same name on the input elements as in the gltf nodes so they could be used to modify its position, add tooltips lines, highlight them and so on

[cvan]

Brandon suggests this Poly model hehe. You can read his full comment here: immersive-web/webxr#319 (comment)

[fernandojsg]

@cvan that's a cool idea actually :) although I still see the need of identify and get the mesh for the specific controller you're using

[blairmacintyre]

Based on the trajectory of the discussions in that immersive-web thread, I suspect we are NOT going to be able to identify the device and controller, and thus won't be able to render the exact controller. From a "webby" perspective, I actually think this makes sense. We don't want people making "only on Vive" experiences ... heck, we're trying to convince folks to not even do "only in VR" ...

[TrevorFSmith]

In the short term we'll have controller IDs on the gamepad APIs, but it does look like we'll have to use more generic source info (including models) for the WebXR device APIs.

I'm all for writing non-hw-specific code, but apps do need to show the user which buttons do what actions and it's lame when the model doesn't match the hw.

[cvan]

@blairmacintyre: yes, @TrevorFSmith is correct. Gamepad objects will continue to have the id property to identify the name of the controller.

there was just a comment from a Chrome engineer who is one of the original spec co-authors about preserving the name (id) even in Gamepad API v2: w3c/gamepad#65 (comment)

the only difference being that the id will not exist as strings with vendor and prefix codes, so you don't have to do silly parsing.

so this …

{"id": "Generic USB Joystick (Vendor: 0079, Product: 0006)"}

becomes this …

{"id": "Generic USB Joystick", "vendor": "0079", "product": "0006"}

[TrevorFSmith]

The new immersive web XR gamepad mappings project will satisfy this so there's no need to build it into action-input.