Allow multiple active sessions at once #236

toji · 2017-05-31T21:39:28Z

Step one in working towards a more robust magic window solution now that
we've decided it should be part of 2.0

This change removes any concepts that assume a device may only have a
single active session at a time. This allows multiple non-exclusive
VRSessions to be active at once, which will be important for enabling
various interesting magic window scenarios in the future. With this
change starting an exclusive session suspends the non-exclusive ones,
but doesn’t end them.

IDL Changes:

Remove VRDevice.activeSession
VRDevice.onsessionchange -> VRSession.onended

Comments welcome! Working on some follow up PRs to this one that should
provide more context if this seems confusing. I wanted to avoid having a
single huge "Magic Window" explainer overhaul, though.

NellWaliczek · 2017-06-02T20:53:13Z

Hey @toji,
I'm curious about what scenario is motivating this change? We'd talked about single vs. multiple sessions before, but I thought we'd come to the conclusion only one was needed? I'm a bit nervous about how this might relate to input routing and security considerations, but if there's something concrete motivating the change let's talk through it!

toji · 2017-06-03T05:51:05Z

@NellWaliczek and I talked directly about this, but to recap for the benefit of the rest of the group:

Motivation for allowing multiple non-exclusive sessions is (mostly) galleries of magic window content in a world where each non-exclusive session is bound to an output canvas as suggested by #237. This is especially important when dealing with a "punch through" magic window scenario, since each canvas will need it's own subtly different projection and view matrices.

More generally, it would be fairly limiting if you could only manage one magic window piece of content per page. Imagine if a third party library creates one without you being aware, at which point your request for a non-exclusive session just starts failing for unknown reasons.

Input routing IS a problematic case, but mostly for inputs that have no implicit ray/direction associated with them: standard gamepads, bluetooth clickers, etc. Touch and mouse inputs have pretty obvious mappings to a specific session/canvas pair, as would using a VR controller to interact with a 2D page viewed in VR. (The pick ray will cast into a specific canvas.)

If Nell wants to fill in any relevant details of our discussion that I've forgotten I'd appreciate it!

toji · 2017-06-15T21:32:33Z

Rebased this against the recent master changes, would still like to land it. Again, the primary motivator is the idea that pages may want multiple magic window elements active at once, and if we only allow a single active session at a time that would both discourage potentially interesting use cases AND force motivated developers to fall back to potentially ugly hacks (like full-screen canvases with carefully scissored content rects. 🤢)

Happy to hear any and all feedback!

toji · 2017-06-22T17:00:49Z

I'm a little concerned about the lack of feedback here (outside of a conversation with Nell), especially given that this PR has been open for 3 weeks. I feel like this is a pretty important change that will ultimately make the API much more useful and would like to land it soon.

Would love to hear from @kearwood, @DigiTec, or @mkeblx for example. Otherwise I guess I'll assume that lack of protest is a positive sign?

kearwood · 2017-06-22T18:20:06Z

Hi @toji ! I've had a chance to review and find this to be a good model, compatible with concepts we have been exploring on our end.

Would it be worth expanding this to explain the behavior of link traversal when changing urls in an active exclusive or non-exclusive session?

Perhaps we could say something to the effect of.. "If the content presenting an exclusive session changes the URL, the destination page is also granted an exclusive session without a user gesture in order to create a seamless link traversal without breaking immersion. If content does not have an active WebVR session or has a non-exclusive WebVR session and changes the URL, a user gesture is required to grant permission to the new page when it requests an exclusive session."

toji · 2017-06-22T19:03:29Z

I don't mind the intent behind the link traversal text you gave. Looking at the entire navigation section we've got right now though reminds me that it's been woefully neglected and needs a much larger overhaul. That should probably be handled in a separate pull request. I can start working on that, keeping your suggestion in mind.

kearwood · 2017-06-22T19:07:42Z

@toji Sounds good to me. I'd be glad to help with the overhaul on the link traversal section.

@kearwood

Prompted by @kearwood’s comments on #236

ddorwin · 2017-06-23T02:22:56Z

explainer.md

-In this case, we only ask if the ability to have `exclusive` access (Which includes the ability to display imagery on the headset). Note that `exclusive: true` is actually the dictionary default, and so does not need to be specified here. It’s made explicit in this example for clarity.
+Sessions can be created with two levels of access:
+
+**Exclusive Access**: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event.


The "Only one" statement needs to be more specific. A single UA could have multiple if there are multiple HMDs.

It can't be per VRDevice because there may be many multiple VRDevice objects in the UA backed by a single HMD. Is how this works described in this doc? If not, does it need to be? On a related note, what happens if there is already an exclusive session and a different document requests exclusive access?

I know we've talked about it off and on in the past, but I don't think I've ever gotten a strong sense of a beneficial scenario where a single piece of hardware is exposed as multiple VRDevices. Did you have something in mind?

Regardless of that, though, you're right that this needs clarification.

I simply meant that multiple frames or documents could call getDevices() and have a VRDevice referencing the same HMD. For my first paragraph, I think we need to say something like there can only be one exclusive session for each underlying device. For the second paragraph, we probably need some text about conflicts. For example, an exclusive request when there is already an active exclusive session for the underlying device somewhere else in the UA (or client) will fail. Maybe there is language in WebUSB or Web Bluetooth that we can use as a model.

Ah, got it. Thanks for the clarification.

ddorwin · 2017-06-23T02:24:25Z

explainer.md

+
+**Exclusive Access**: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event.
+
+**Non-Exclusive Access**: Requested with the `exclusive: false` dictionary argument. Non-exclusive sessions do not have the ability to display anything on the `VRDevice`, but are able to access device tracking information and use it to render content on the page. This technique, where a scene rendered to the page is responsive to device movement is sometimes referred to as "Magic Window" mode. It's especially useful for mobile devices, where moving the device can be used to look around a scene. Devices like Tango phones and tablets with 6DoF tracking capabilities may expose them via non-exclusive sessions even if the hardware is not capable of immersive, stereo presentation. Multiple non-exclusive sessions can be active at once, but all non-exclusive sessions are suspended when an exclusive session is active. Non-exclusive sessions are not required to be created within a user gesture event.


Similarly, how are requests from multiple documents across the UA handled?

ddorwin · 2017-06-23T02:28:41Z

explainer.md

+
+**Exclusive Access**: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event.
+
+**Non-Exclusive Access**: Requested with the `exclusive: false` dictionary argument. Non-exclusive sessions do not have the ability to display anything on the `VRDevice`, but are able to access device tracking information and use it to render content on the page. This technique, where a scene rendered to the page is responsive to device movement is sometimes referred to as "Magic Window" mode. It's especially useful for mobile devices, where moving the device can be used to look around a scene. Devices like Tango phones and tablets with 6DoF tracking capabilities may expose them via non-exclusive sessions even if the hardware is not capable of immersive, stereo presentation. Multiple non-exclusive sessions can be active at once, but all non-exclusive sessions are suspended when an exclusive session is active. Non-exclusive sessions are not required to be created within a user gesture event.


What restrictions are there on which frames can request non-exclusive access at the same time? Are multiple iframes allowed to request access at once? Must they be same-origin to each other or the top-level document?

Requiring that the frame receiving poses currently has focus is an important privacy mitigation, especially because there is no gesture requirement. Allowing multiple frames to receive poses could be problematic.

This definitely needs to be expanded on. I'll add a section dedicated to session security.

ddorwin · 2017-06-23T02:36:21Z

explainer.md

+
+**Exclusive Access**: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event.
+
+**Non-Exclusive Access**: Requested with the `exclusive: false` dictionary argument. Non-exclusive sessions do not have the ability to display anything on the `VRDevice`, but are able to access device tracking information and use it to render content on the page. This technique, where a scene rendered to the page is responsive to device movement is sometimes referred to as "Magic Window" mode. It's especially useful for mobile devices, where moving the device can be used to look around a scene. Devices like Tango phones and tablets with 6DoF tracking capabilities may expose them via non-exclusive sessions even if the hardware is not capable of immersive, stereo presentation. Multiple non-exclusive sessions can be active at once, but all non-exclusive sessions are suspended when an exclusive session is active. Non-exclusive sessions are not required to be created within a user gesture event.


nit: Remove the comma before "but."

ddorwin · 2017-06-23T02:42:19Z

explainer.md

-In this case, we only ask if the ability to have `exclusive` access (Which includes the ability to display imagery on the headset). Note that `exclusive: true` is actually the dictionary default, and so does not need to be specified here. It’s made explicit in this example for clarity.
+Sessions can be created with two levels of access:
+
+**Exclusive Access**: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event.


It is almost certain that a user gesture event will not be required in some cases, such as the activate handler. The current explainer text says, "The activate event acts as a user gesture..." I don't think we want to treat that as a generic gesture as has previously been discussed. Should we file a separate bug to clean up gesture-related language throughout?

Good point, and yes. That's probably worth it's own bug.

I filed #248.

ddorwin · 2017-06-23T02:44:33Z

explainer.md

  // Ending the session stops executing callbacks passed to requestFrame().
  // To continue rendering, use the window's AnimationFrame callback
  window.requestAnimationFrame(onDrawFrame);
 }
 ```

-In addition to the application ending the session manually, the UA may force the session to end at any time for a variety of reasons. Well behaved applications should monitor the `sessionchange` event on the `VRDevice` to detect when that happens.
+In addition to the application ending a session manually, the UA may force sessions to end at any time for a variety of reasons. Well behaved applications should monitor the `ended` event on the `VRSession` to detect when that happens.


Maybe this is where some of the scenarios I mentioned above should be covered.

Agreed, but probably as a new PR.

ddorwin · 2017-06-23T02:45:19Z

explainer.md

  }
 });
 ```

 ### Responding to a suspended session

-The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the headset may be less accurate. The UA is expected to present a tracked environment to the user when the page is being throttled to prevent user discomfort.
+The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. Additionally, non-exclusive sessions are suspended while an exclusive session is active.


security or privacy risk

ddorwin · 2017-06-23T02:48:40Z

explainer.md

-The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the headset may be less accurate. The UA is expected to present a tracked environment to the user when the page is being throttled to prevent user discomfort.
+The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. Additionally, non-exclusive sessions are suspended while an exclusive session is active.
+
+While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the device may be less accurate. If the user is wearing a headset the UA is expected to present a tracked environment when the page is being throttled to prevent user discomfort.


Should this be normatively specified? It could be problematic if some UAs provide throttled poses but others provide none. Also, it may be useful to provide clear privacy/security guidance in the spec. This will at least ensure the implementers have the benefit of the collective community.

I think it's going to depend on the hardware/API/OS capabilities and the privacy needs. I'd like to address this more directly, but it's a big enough topic that it's probably worth a whole new issue and/or PR.

Okay, please file the new issue since you have a better idea of what it should be.

ddorwin · 2017-06-23T02:49:07Z

explainer.md

-The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the headset may be less accurate. The UA is expected to present a tracked environment to the user when the page is being throttled to prevent user discomfort.
+The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. Additionally, non-exclusive sessions are suspended while an exclusive session is active.
+
+While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the device may be less accurate. If the user is wearing a headset the UA is expected to present a tracked environment when the page is being throttled to prevent user discomfort.


Explain what a tracked environment is.

ddorwin · 2017-06-23T02:51:15Z

explainer.md

-  if (vrDevice.activeSession) {
-    vrDevice.activeSession.endSession().then(OnSessionEnded);
+  if (vrSession) {
+    vrSession.endSession().then(OnSessionEnded);
  }
 });
 ```

 ### Responding to a suspended session


It may help address comments above and below to add a separate section that discusses when a UA should suspend a session. That could be referenced both above and from here.

toji · 2017-06-23T21:40:35Z

Thanks for all the feedback @ddorwin! Addressed a bunch of it, but I've deferred some of the security-centric additions to a different PR for the sake of easier review.

ddorwin

Thanks. Just some minor comments.

ddorwin · 2017-06-23T22:00:36Z

explainer.md

 The first thing that any VR-enabled page will want to do is enumerate the available VR hardware and, if present, advertise VR functionality to the user.

-[`navigator.vr.getDevices`](https://w3c.github.io/webvr/#navigator-getvrdevices-attribute) returns a [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a list of available devices. Each [`VRDevice`](https://w3c.github.io/webvr/#interface-vrdevice) represents a physical unit of VR hardware that can present imagery to the user somehow. On desktops this will usually be a headset peripheral; on mobile devices it may represent the device itself in conjunction with a viewer harness (e.g., Google Cardboard or Samsung Gear VR). It may also represent devices without stereo presentation capabilities but more advanced tracking, such as Tango devices.
+[`navigator.vr.getDevices`](https://w3c.github.io/webvr/#navigator-getvrdevices-attribute) returns a [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a list of available devices. Each [`VRDevice`](https://w3c.github.io/webvr/#interface-vrdevice) represents a physical unit of VR hardware that can present imagery to the user somehow, referred to here as a "VR hardware device". On desktops this will usually be a headset peripheral; on mobile devices it may represent the device itself in conjunction with a viewer harness (e.g., Google Cardboard or Samsung Gear VR). It may also represent devices without stereo presentation capabilities but more advanced tracking, such as Tango devices.


Separate issue: These links don't work anymore. Why are they absolute? I guess ../../spec/latest/ might work.

ddorwin · 2017-06-23T22:02:05Z

explainer.md

 The first thing that any VR-enabled page will want to do is enumerate the available VR hardware and, if present, advertise VR functionality to the user.

-[`navigator.vr.getDevices`](https://w3c.github.io/webvr/#navigator-getvrdevices-attribute) returns a [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a list of available devices. Each [`VRDevice`](https://w3c.github.io/webvr/#interface-vrdevice) represents a physical unit of VR hardware that can present imagery to the user somehow. On desktops this will usually be a headset peripheral; on mobile devices it may represent the device itself in conjunction with a viewer harness (e.g., Google Cardboard or Samsung Gear VR). It may also represent devices without stereo presentation capabilities but more advanced tracking, such as Tango devices.
+[`navigator.vr.getDevices`](https://w3c.github.io/webvr/#navigator-getvrdevices-attribute) returns a [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a list of available devices. Each [`VRDevice`](https://w3c.github.io/webvr/#interface-vrdevice) represents a physical unit of VR hardware that can present imagery to the user somehow, referred to here as a "VR hardware device". On desktops this will usually be a headset peripheral; on mobile devices it may represent the device itself in conjunction with a viewer harness (e.g., Google Cardboard or Samsung Gear VR). It may also represent devices without stereo presentation capabilities but more advanced tracking, such as Tango devices.


nits:
"On desktop clients,"
"on mobile devices,"

ddorwin · 2017-06-23T22:04:39Z

explainer.md

 The first thing that any VR-enabled page will want to do is enumerate the available VR hardware and, if present, advertise VR functionality to the user.

-[`navigator.vr.getDevices`](https://w3c.github.io/webvr/#navigator-getvrdevices-attribute) returns a [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a list of available devices. Each [`VRDevice`](https://w3c.github.io/webvr/#interface-vrdevice) represents a physical unit of VR hardware that can present imagery to the user somehow. On desktops this will usually be a headset peripheral; on mobile devices it may represent the device itself in conjunction with a viewer harness (e.g., Google Cardboard or Samsung Gear VR). It may also represent devices without stereo presentation capabilities but more advanced tracking, such as Tango devices.
+[`navigator.vr.getDevices`](https://w3c.github.io/webvr/#navigator-getvrdevices-attribute) returns a [`Promise`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a list of available devices. Each [`VRDevice`](https://w3c.github.io/webvr/#interface-vrdevice) represents a physical unit of VR hardware that can present imagery to the user somehow, referred to here as a "VR hardware device". On desktops this will usually be a headset peripheral; on mobile devices it may represent the device itself in conjunction with a viewer harness (e.g., Google Cardboard or Samsung Gear VR). It may also represent devices without stereo presentation capabilities but more advanced tracking, such as Tango devices.


Does the spec currently require that all VRDevices "can present imagery to the user somehow"? For example, in a pure magic window case where there is no headset, the VRDevice is not actually capable of presenting imagery.

Hmm... I think that's probably a matter of semantics. (I tend to think of Magic Window as "Presenting imagery to the user") but the point stands that this is confusing.

Ah, I do recall that we wanted to call this out somehow to differentiate a VR headset from a VR controller. Not sure how to mitigate that...

Speaking of semantics, is a phone "a physical unit of VR hardware?" ;)

ddorwin · 2017-06-23T22:06:13Z

explainer.md

-If a `VRDevice` is available and has the appropriate capabilities the page will usually want to add some UI to trigger activation of "VR Presentation Mode", where the page can begin sending imagery to the device. Testing to see if the device supports the capabilities the page needs is done via the `supportsSession` call, which takes a dictionary of the desired functionality and returns whether or not the device can create a session supporting them. Querying for support this way is necessary because it allows the application to detect what VR features are available without actually engaging the sensors or beginning presentation, which can incur significant power or performance overhead on some systems and may have side effects such as launching a VR status tray or storefront.
+### Sessions
+
+A `VRDevice` indicates the presence of a VR hardware device but provides very little information about it outside of a name that could be used to select it from a list. In order to do anything that involves the hardware's presentation or tracking capabilities the page will need to request a [`VRSession`](https://w3c.github.io/webvr/#interface-vrsession) from the `VRDevice`.


It's nice to somehow indicate that you are referencing a definition and link to that definition. This may not be easy to do in MD, but in the spec, you'd probably want to make "VR hardware device" a link.

ddorwin · 2017-06-23T22:07:55Z

explainer.md

-If a `VRDevice` is available and has the appropriate capabilities the page will usually want to add some UI to trigger activation of "VR Presentation Mode", where the page can begin sending imagery to the device. Testing to see if the device supports the capabilities the page needs is done via the `supportsSession` call, which takes a dictionary of the desired functionality and returns whether or not the device can create a session supporting them. Querying for support this way is necessary because it allows the application to detect what VR features are available without actually engaging the sensors or beginning presentation, which can incur significant power or performance overhead on some systems and may have side effects such as launching a VR status tray or storefront.
+### Sessions
+
+A `VRDevice` indicates the presence of a VR hardware device but provides very little information about it outside of a name that could be used to select it from a list. In order to do anything that involves the hardware's presentation or tracking capabilities the page will need to request a [`VRSession`](https://w3c.github.io/webvr/#interface-vrsession) from the `VRDevice`.


nits: "...capabilities, the application must request..."

I believe "application" should be used instead of "page" in nearly all cases. This can be addressed separately.

ddorwin · 2017-06-23T22:08:43Z

explainer.md

-If a `VRDevice` is available and has the appropriate capabilities the page will usually want to add some UI to trigger activation of "VR Presentation Mode", where the page can begin sending imagery to the device. Testing to see if the device supports the capabilities the page needs is done via the `supportsSession` call, which takes a dictionary of the desired functionality and returns whether or not the device can create a session supporting them. Querying for support this way is necessary because it allows the application to detect what VR features are available without actually engaging the sensors or beginning presentation, which can incur significant power or performance overhead on some systems and may have side effects such as launching a VR status tray or storefront.
+### Sessions
+
+A `VRDevice` indicates the presence of a VR hardware device but provides very little information about it outside of a name that could be used to select it from a list. In order to do anything that involves the hardware's presentation or tracking capabilities the page will need to request a [`VRSession`](https://w3c.github.io/webvr/#interface-vrsession) from the `VRDevice`.

 Sessions can be created with two levels of access:


nit: "one of two"

ddorwin · 2017-06-23T22:13:53Z

explainer.md

+
+### Detecting and advertising VR mode
+
+If a `VRDevice` is available and able to create an exclusive session the page will usually want to add some UI to trigger activation of "VR Presentation Mode", where the page can begin sending imagery to the device. Testing to see if the device supports the capabilities the page needs is done via the `supportsSession` call, which takes a dictionary of the desired functionality and returns whether or not the device can create a session supporting them. Querying for support this way is necessary because it allows the application to detect what VR features are available without actually engaging the sensors or beginning presentation, which can incur significant power or performance overhead on some systems and may have side effects such as launching a VR status tray or storefront.


nit: "...session, the application..."

ddorwin · 2017-06-23T22:17:03Z

explainer.md

+
+**Non-Exclusive Access**: Requested with the `exclusive: false` dictionary argument. Non-exclusive sessions do not have the ability to display anything on the `VRDevice`, but are able to access device tracking information and use it to render content on the page. This technique, where a scene rendered to the page is responsive to device movement is sometimes referred to as "Magic Window" mode. It's especially useful for mobile devices, where moving the device can be used to look around a scene. Devices like Tango phones and tablets with 6DoF tracking capabilities may expose them via non-exclusive sessions even if the hardware is not capable of immersive, stereo presentation. Multiple non-exclusive sessions can be active at once, but all non-exclusive sessions are suspended when an exclusive session is active. Non-exclusive sessions are not required to be created within a user gesture event.
+
+In this case, we ask if the `VRDevice` supports sessions with `exclusive` access, since we want the ability to display imagery on the headset.


This comment was resolved by moving the text above.

Does it make sense to say "In the following example" instead? "In this case" usually refer to something in the preceding text.

ddorwin · 2017-06-23T22:20:20Z

explainer.md

@@ -105,7 +111,7 @@ async function OnVRAvailable() {

 ### Beginning a VR session

-Clicking that button will attempt to initiate a [`VRSession`](https://w3c.github.io/webvr/#interface-vrsession), which manages input and output for the display. When creating a session with `VRDevice.requestSession` the capabilities that the returned session must have are passed in via a dictionary, exactly like the `supportsSession` call. If `supportsSession` returned true for a given dictionary then calling `requestSession` with the same dictionary values should be reasonably expected to succeed, barring external factors (such as `requestSession` not being called in a user gesture for an exclusive session or another page currently having an exclusive session for the same device.)
+Clicking that button will attempt to acquire a `VRSession` by callling `VRDisplay.requestSession`. This returns a promise that resolves to a `VRSession` upon success. When requesting a session the capabilities that the returned session must have are passed in via a dictionary, exactly like the `supportsSession` call. If `supportsSession` returned true for a given dictionary then calling `requestSession` with the same dictionary values should be reasonably expected to succeed, barring external factors (such as `requestSession` not being called in a user gesture for an exclusive session or another page currently having an exclusive session for the same device.)


You might want to use "VR hardware device" at the end.

nits: Commas after "When requesting a session" and before "then."

toji · 2017-06-23T23:19:00Z

Made some more updates to take into account @ddorwin's continued feedback. (Thanks!) Didn't strip out URLs yet or do full-doc page->application replace. Would prefer to do those separately.

NellWaliczek

Most of these changes look good. I'm mostly curious about a few things:

Why suspend rather than end non-exclusive sessions when an exclusive session takes over?
What is the expected behavior of a non-exclusive session when tracking data becomes available (say because another app is utilizing the VR system)?
I'm still torn about the multiple non-exclusive sessions... It's not that I don't think it could be helpful. I'm just wondering if it's necessary for 2.0 or would be better as a 2.1 feature. We could totally set ourselves up for this by removing the "activeSession" property now though! I guess it comes down to the urgency of this functionality... is there a developer specifically asking for this right now?

NellWaliczek · 2017-06-27T19:56:42Z

explainer.md

+
+Sessions can be created with one of two levels of access:
+
+**Exclusive Access**: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session per VR hardware device is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event or within another callback that has been explicitly indicated to allow exclusive session creation.


It might be good to give an example of what sort of callback would allow this (I know we'll get into more detail in navigation PR, so maybe it can just wait until then...)

NellWaliczek · 2017-06-27T19:59:56Z

explainer.md

-The page may also want to create a session that doesn't need exclusive access to the device for tracking purposes. Since the `VRSession` is also what provides access to the device's position and orientation data requesting a non-exclusive session enables what's referred to as "Magic Window" use cases, where the scene is rendered on the page normally but is responsive to device movement. This is especially useful for mobile devices, where moving the device can be used to look around a scene. Devices with Tango tracking capabilities may also expose 6DoF tracking this way, even when the device itself is not capable of stereo presentation.
-
-Requesting a new type of session will end any previously active ones.
+Clicking that button will attempt to acquire a `VRSession` by callling `VRDisplay.requestSession`. This returns a promise that resolves to a `VRSession` upon success. When requesting a session, the capabilities that the returned session must have are passed in via a dictionary, exactly like the `supportsSession` call. If `supportsSession` returned true for a given dictionary, then calling `requestSession` with the same dictionary values should be reasonably expected to succeed, barring external factors (such as `requestSession` not being called in a user gesture for an exclusive session or another application currently having an exclusive session for the same VR hardware device.)


"Clicking that button" seems like a bit of a non sequitur... perhaps "clicking the enterVrButton in the previous sample"?

NellWaliczek · 2017-06-27T20:01:29Z

explainer.md

-The page may also want to create a session that doesn't need exclusive access to the device for tracking purposes. Since the `VRSession` is also what provides access to the device's position and orientation data requesting a non-exclusive session enables what's referred to as "Magic Window" use cases, where the scene is rendered on the page normally but is responsive to device movement. This is especially useful for mobile devices, where moving the device can be used to look around a scene. Devices with Tango tracking capabilities may also expose 6DoF tracking this way, even when the device itself is not capable of stereo presentation.
-
-Requesting a new type of session will end any previously active ones.
+Clicking that button will attempt to acquire a `VRSession` by callling `VRDisplay.requestSession`. This returns a promise that resolves to a `VRSession` upon success. When requesting a session, the capabilities that the returned session must have are passed in via a dictionary, exactly like the `supportsSession` call. If `supportsSession` returned true for a given dictionary, then calling `requestSession` with the same dictionary values should be reasonably expected to succeed, barring external factors (such as `requestSession` not being called in a user gesture for an exclusive session or another application currently having an exclusive session for the same VR hardware device.)


"another application currently having an exclusive session for the same VR hardware device" isn't true for all VR platforms. :) Can we reword this to be a bit clearer by perhaps removing this reason and just saying that it's up to the underlying platform to decide if it can honor the request?

NellWaliczek · 2017-06-27T20:02:24Z

explainer.md

-        // May fail for a variety of reasons, including another page already
-        // having exclusive access to the device. Probably just want to render
-        // the scene normally without any tracking at this point.
+        // May fail for a variety of reasons, including another application


Same comment as above

NellWaliczek · 2017-06-27T20:07:44Z

explainer.md

  }
 });
 ```

 ### Responding to a suspended session

-The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the headset may be less accurate. The UA is expected to present a tracked environment to the user when the page is being throttled to prevent user discomfort.
+The UA may temporarily suspend a session if allowing the page to continue reading the headset position represents a security or privacy risk (like when the user is entering a password or URL with a virtual keyboard, in which case the head motion may infer the user's input), or if other content is obscuring the page's output. Additionally, non-exclusive sessions are suspended while an exclusive session is active.


Why suspend non-exclusive sessions rather than end them?

Thinking about the SketchFab gallery scenario: if you have a grid of magic windows, then click on one of them to jump into immersive mode, then exit again, you ideally want to be back to the same grid of magic windows. That's still do-able if the sessions end upon creating an exclusive session, but it's much more of a pain for page authors. Suspensions keeps the basic wiring around without requiring that the frame loop be pumped.

NellWaliczek · 2017-06-27T20:09:29Z

explainer.md


-In general the page should continue drawing frames in response to `VRSession.requestFrame()` callbacks. The UA may use these frames as part of it's tracked environment, though they may be partially occluded, blurred, or otherwise manipulated. Still, some applications may wish to respond to this suspension by halting game logic, purposefully obscuring content, or pausing media. To do so, the application should listen for the `blur` and `focus` events from the `VRSession`. For example, a 360 media player would do this to pause the video/audio whenever the UA has obscured it.
+While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the device may be less accurate. If the user is wearing a headset the UA is expected to present a tracked environment (a scene which remains responsive to user's head motion) when the page is being throttled to prevent user discomfort.


What does this mean in the context of a suspended session? "If the user is wearing a headset the UA is expected to present a tracked environment (a scene which remains responsive to user's head motion) when the page is being throttled to prevent user discomfort."

This change removes any concepts that assume a device may only have a single active session at a time. This allows multiple non-exclusive VRSessions to be active at once, which will be important for enabling various interesting magic window scenarios in the future. With this change starting an exclusive session suspends the non-exclusive ones, but doesn’t end them. IDL Changes: - Remove VRDevice.activeSession - VRDevice.onsessionchange -> VRSession.onended

Wanted to explicitly state the differences between the two modes and why you’d want to use one or the other.

Some security discussion has been deferred to a follow up PR for the sake of readability.

Following discussion on implementor call, for the moment we want to just remove elements of the explainer that would explicitly block multiple non-exclusive sessions at once (like the activeSession attribute) while not explicitly stating that multiple are support just yet. (Might be a 2.1 feature? We’ll see.)

toji · 2017-06-28T04:53:13Z

In the WebVR implementors call today we agreed that it would be good to get the majority of the content from this PR merged in, but defer the bit that explicitly states that multiple non-exclusive sessions are allowed at once since there's still some questions around that. We should however keep the removal of elements that would explicitly prevent that in the future (such as the activeSession attribute.) I've made that change, and am merging now.

Next, a followup PR should focus on clearly defining the terms "suspended" and "ended" and the conditions under which a session could be either suspended or ended.

toji added this to the 2.0 milestone May 31, 2017

toji mentioned this pull request Jun 13, 2017

Detail magic window support #237

Closed

toji force-pushed the multi_session branch from 91f7a39 to f113613 Compare June 15, 2017 21:16

toji added a commit that referenced this pull request Jun 22, 2017

First stab at updating the explainer's navigation section

fa8dc45

Prompted by @kearwood’s comments on #236

toji mentioned this pull request Jun 22, 2017

Update navigation logic #247

Closed

ddorwin reviewed Jun 23, 2017

View reviewed changes

ddorwin approved these changes Jun 23, 2017

View reviewed changes

NellWaliczek reviewed Jun 27, 2017

View reviewed changes

toji added 5 commits June 27, 2017 21:02

Clarify the difference between Exclusive and Non-Exclusive modes

6895ee1

Wanted to explicitly state the differences between the two modes and why you’d want to use one or the other.

Updated in response to multiple suggestions from @ddorwin

3844bd1

Some security discussion has been deferred to a follow up PR for the sake of readability.

More changes from more @ddorwin feedback.

7dd79ef

Incorporating some more feedback from @NellWaliczek

2456728

toji force-pushed the multi_session branch from 565c439 to 2456728 Compare June 28, 2017 04:04

toji merged commit df0afd3 into master Jun 28, 2017

toji deleted the multi_session branch June 29, 2017 18:59

toji mentioned this pull request Jun 29, 2017

Clarify the definitions and uses of "suspended" vs. "ended" sessions #252

Merged

cwilso modified the milestones: Spec-Complete for 1.0, 1.0 Apr 30, 2019


		Exclusive Access: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event.

		Non-Exclusive Access: Requested with the `exclusive: false` dictionary argument. Non-exclusive sessions do not have the ability to display anything on the `VRDevice`, but are able to access device tracking information and use it to render content on the page. This technique, where a scene rendered to the page is responsive to device movement is sometimes referred to as "Magic Window" mode. It's especially useful for mobile devices, where moving the device can be used to look around a scene. Devices like Tango phones and tablets with 6DoF tracking capabilities may expose them via non-exclusive sessions even if the hardware is not capable of immersive, stereo presentation. Multiple non-exclusive sessions can be active at once, but all non-exclusive sessions are suspended when an exclusive session is active. Non-exclusive sessions are not required to be created within a user gesture event.


		### Detecting and advertising VR mode

		If a `VRDevice` is available and able to create an exclusive session the page will usually want to add some UI to trigger activation of "VR Presentation Mode", where the page can begin sending imagery to the device. Testing to see if the device supports the capabilities the page needs is done via the `supportsSession` call, which takes a dictionary of the desired functionality and returns whether or not the device can create a session supporting them. Querying for support this way is necessary because it allows the application to detect what VR features are available without actually engaging the sensors or beginning presentation, which can incur significant power or performance overhead on some systems and may have side effects such as launching a VR status tray or storefront.


		Non-Exclusive Access: Requested with the `exclusive: false` dictionary argument. Non-exclusive sessions do not have the ability to display anything on the `VRDevice`, but are able to access device tracking information and use it to render content on the page. This technique, where a scene rendered to the page is responsive to device movement is sometimes referred to as "Magic Window" mode. It's especially useful for mobile devices, where moving the device can be used to look around a scene. Devices like Tango phones and tablets with 6DoF tracking capabilities may expose them via non-exclusive sessions even if the hardware is not capable of immersive, stereo presentation. Multiple non-exclusive sessions can be active at once, but all non-exclusive sessions are suspended when an exclusive session is active. Non-exclusive sessions are not required to be created within a user gesture event.

		In this case, we ask if the `VRDevice` supports sessions with `exclusive` access, since we want the ability to display imagery on the headset.


		Sessions can be created with one of two levels of access:

		Exclusive Access: The default mode, but can be explicitly requested with the `exclusive: true` dictionary argument. Exclusive sessions present content directly to the `VRDevice`, enabling immersive VR presentation. Only one exclusive session per VR hardware device is allowed at a time across the entire UA. Exclusive sessions must be created within a user gesture event or within another callback that has been explicitly indicated to allow exclusive session creation.


		In general the page should continue drawing frames in response to `VRSession.requestFrame()` callbacks. The UA may use these frames as part of it's tracked environment, though they may be partially occluded, blurred, or otherwise manipulated. Still, some applications may wish to respond to this suspension by halting game logic, purposefully obscuring content, or pausing media. To do so, the application should listen for the `blur` and `focus` events from the `VRSession`. For example, a 360 media player would do this to pause the video/audio whenever the UA has obscured it.
		While suspended the page may either refresh the vr device at a slower rate or not at all, and poses queried from the device may be less accurate. If the user is wearing a headset the UA is expected to present a tracked environment (a scene which remains responsive to user's head motion) when the page is being throttled to prevent user discomfort.

Allow multiple active sessions at once #236

Allow multiple active sessions at once #236

Conversation

toji commented May 31, 2017

NellWaliczek commented Jun 2, 2017

toji commented Jun 3, 2017

toji commented Jun 15, 2017

toji commented Jun 22, 2017

kearwood commented Jun 22, 2017

toji commented Jun 22, 2017

kearwood commented Jun 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toji commented Jun 23, 2017

ddorwin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toji commented Jun 23, 2017

NellWaliczek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toji commented Jun 28, 2017