Fill out section on trusted UI #875

Manishearth · 2019-10-14T23:42:16Z

/fixes #718, /fixes #719

Unsure of this approach, still working on things.

avadacatavra · 2019-10-16T14:16:54Z

index.bs

+  - Pausing the immersive session and showing some form of desktop environment in which a prompt can be shown
+
+</div>
+


Did we also want to enumerate the properties of a trusted UI from #718

Manishearth · 2019-10-16T14:37:08Z

Ah, yes I definitely should do that, thanks

…

On Wed, Oct 16, 2019, 7:17 AM Diane Hosfelt ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In index.bs <#875 (comment)>: > Trusted Environment {#trustedenvironment-security} ------------------- -If the virtual environment does not consistently track the user's head motion with low latency and at a high frame rate the user may become disoriented or physically ill. Since it is impossible to force pages to produce consistently performant and correct content the user agent MUST provide a tracked, trusted environment and an [=XR Compositor=] which runs asynchronously from page content. The compositor is responsible for compositing the trusted and untrusted content. If content is not performant, does not submit frames, or terminates unexpectedly the user agent should be able to continue presenting a responsive, trusted UI. +The user agent MUST support showing a <dfn>Trusted UI</dfn>, that is, an interface that the user can trust comes from the user agent, which the user may interact with without interference from the page. Some form of [=trusted UI=] MUST be used to show permissions prompts. + + +A [=trusted UI=] which does not exit immersive mode is known as a <dfn>trusted immersive UI</dfn>. User agents are not required to support [=trusted immersive UI=], they may instead temporarily pause/exit immersive mode to handle prompts to the user. + +<div class="note"> +Note: Examples of [=trusted UI=] include: + - The default 2D mode browser shown when not in immersive mode + - A prompt shown within immersive mode which can only be interacted with via a reserved hardware button to prevent spoofing + - Pausing the immersive session and showing some form of desktop environment in which a prompt can be shown + +</div> + Did we also want to enumerate the properties of a trusted UI from #718 <#718> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#875?email_source=notifications&email_token=AAMK6SEDNAY23A7ACVMJPATQO4O7LA5CNFSM4JAVIL7KYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCIEXYLI#pullrequestreview-302611501>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMK6SD3R6LLT3LS7U4CSD3QO4O7LANCNFSM4JAVIL7A> .

toji · 2019-10-17T21:17:22Z

index.bs

+Note: Examples of [=trusted UI=] include:
+  - The default 2D mode browser shown when not in immersive mode
+  - A prompt shown within immersive mode which can only be interacted with via a reserved hardware button to prevent spoofing
+  - Pausing the immersive session and showing some form of desktop environment in which a prompt can be shown


Pausing the immersive session and showing some form of desktop environment in which a prompt can be shown

I feel like the phrasing on this one is maybe a tad bit too restrictive? Specifically the use of the phrase "desktop environment", which seems to suggest a very specific and potentially mobile-excluding concept.

What about something along the lines of this?

Pausing the immersive session to show a prompt in the native system environment.

NellWaliczek

A handful of miscellaneous feedback. This is a thorny bit of spec text to get right and I appreciate you digging in!

NellWaliczek · 2019-10-18T21:10:09Z

index.bs


 Trusted Environment {#trustedenvironment-security}
 -------------------

-If the virtual environment does not consistently track the user's head motion with low latency and at a high frame rate the user may become disoriented or physically ill. Since it is impossible to force pages to produce consistently performant and correct content the user agent MUST provide a tracked, trusted environment and an [=XR Compositor=] which runs asynchronously from page content. The compositor is responsible for compositing the trusted and untrusted content. If content is not performant, does not submit frames, or terminates unexpectedly the user agent should be able to continue presenting a responsive, trusted UI.
+The user agent MUST support showing a <dfn>Trusted UI</dfn>, that is, an interface that the user can trust comes from the user agent, which the user may interact with without interference from the page. Some form of [=trusted UI=] MUST be used to show permissions prompts.


It might be a good idea to put a couple sentences before this elaborating on why trusted UI is a harder problem in immersive experiences (i.e. all pixels can be drawn by the developer which allows for spoofing).

NellWaliczek · 2019-10-18T21:12:18Z

index.bs

+
+If the virtual environment does not consistently track the user's head motion with low latency and at a high frame rate the user may become disoriented or physically ill. Since it is impossible to force pages to produce consistently performant and correct content the user agent MUST provide a tracked, trusted environment and an [=XR Compositor=] which runs asynchronously from page content. The compositor is responsible for compositing the trusted and untrusted content. If content is not performant, does not submit frames, or terminates unexpectedly the user agent should be able to continue presenting a responsive, [=trusted UI=].
+
+Additionally, page content has the ability to make users uncomfortable in ways not related to performance. Badly applied tracking, strobing colors, and content intended to offend, frighten, or intimidate are examples of content which may cause the user to want to quickly exit the XR experience. Removing the XR device in these cases may not always be a fast or practical option. To accommodate this the user agent SHOULD provide users with an action, such as pressing a reserved hardware button or performing a gesture, that escapes out of WebXR content and displays the user agent's [=trusted UI=].

 {{XRSession}}s MUST have their [=visibility state=] set to {{XRVisibilityState/"hidden"}} when the user is interacting with potentially sensitive UI from the user agent (such as entering a URL) in the trusted environment.


Is this a duplicate of the last paragraph in the gaze tracking section?

NellWaliczek · 2019-10-18T21:13:36Z

index.bs


 Trusted Environment {#trustedenvironment-security}
 -------------------

-If the virtual environment does not consistently track the user's head motion with low latency and at a high frame rate the user may become disoriented or physically ill. Since it is impossible to force pages to produce consistently performant and correct content the user agent MUST provide a tracked, trusted environment and an [=XR Compositor=] which runs asynchronously from page content. The compositor is responsible for compositing the trusted and untrusted content. If content is not performant, does not submit frames, or terminates unexpectedly the user agent should be able to continue presenting a responsive, trusted UI.
+The user agent MUST support showing a <dfn>Trusted UI</dfn>, that is, an interface that the user can trust comes from the user agent, which the user may interact with without interference from the page. Some form of [=trusted UI=] MUST be used to show permissions prompts.


Nit: there's something about this sentence that seems somewhat oddly phrased, but I can't put my finger on how it might be clearer.

The user agent MUST support showing a Trusted UI, that is, an interface that the user can trust comes from the user agent, which the user may interact with without interference from the page.

index.bs

NellWaliczek · 2019-10-18T21:26:44Z

index.bs

+
+
+
+If the virtual environment does not consistently track the user's head motion with low latency and at a high frame rate the user may become disoriented or physically ill. Since it is impossible to force pages to produce consistently performant and correct content the user agent MUST provide a tracked, trusted environment and an [=XR Compositor=] which runs asynchronously from page content. The compositor is responsible for compositing the trusted and untrusted content. If content is not performant, does not submit frames, or terminates unexpectedly the user agent should be able to continue presenting a responsive, [=trusted UI=].


This seems unrelated enough to trusted ui that it probably warrants its own section heading.

NellWaliczek · 2019-10-18T21:28:18Z

index.bs

+Gaze Tracking {#gazetracking-security}
+-------------
+
+While the API does not yet expose eye tracking capabilities a lot can be inferred about where the user is looking by tracking the orientation of their head. This is especially true of XR devices that have limited input capabilities, such as Google Cardboard, which frequently require users to control a "gaze cursor" with their head orientation. This means that it may be possible for a malicious page to infer what a user is typing on a virtual keyboard or how they are interacting with a virtual UI based solely on monitoring their head movements. For example: if not prevented from doing so a page could estimate what URL a user is entering into the user agent's URL bar.


Is this text still helpful?

While the API does not yet expose eye tracking capabilities a lot can be inferred about where the user is looking by tracking the orientation of their head. This is especially true of XR devices that have limited input capabilities, such as Google Cardboard, which frequently require users to control a "gaze cursor" with their head orientation.

NellWaliczek · 2019-10-18T21:29:10Z

index.bs

+Gaze Tracking {#gazetracking-security}
+-------------
+
+While the API does not yet expose eye tracking capabilities a lot can be inferred about where the user is looking by tracking the orientation of their head. This is especially true of XR devices that have limited input capabilities, such as Google Cardboard, which frequently require users to control a "gaze cursor" with their head orientation. This means that it may be possible for a malicious page to infer what a user is typing on a virtual keyboard or how they are interacting with a virtual UI based solely on monitoring their head movements. For example: if not prevented from doing so a page could estimate what URL a user is entering into the user agent's URL bar.


This text can probably be repurposed as the intro paragraph if it is adjusted slightly.

This means that it may be possible for a malicious page to infer what a user is typing on a virtual keyboard or how they are interacting with a virtual UI based solely on monitoring their head movements. For example: if not prevented from doing so a page could estimate what URL a user is entering into the user agent's URL bar.

NellWaliczek · 2019-10-18T21:29:31Z

index.bs

+
+While the API does not yet expose eye tracking capabilities a lot can be inferred about where the user is looking by tracking the orientation of their head. This is especially true of XR devices that have limited input capabilities, such as Google Cardboard, which frequently require users to control a "gaze cursor" with their head orientation. This means that it may be possible for a malicious page to infer what a user is typing on a virtual keyboard or how they are interacting with a virtual UI based solely on monitoring their head movements. For example: if not prevented from doing so a page could estimate what URL a user is entering into the user agent's URL bar.
+
+To prevent this risk the user agent MUST set the [=visibility state=] of all {{XRSession}}s to {{XRVisibilityState/"hidden"}} when the user is interacting with sensitive, trusted UI such as URL bars or system dialogs. Additionally, to prevent a malicious page from being able to monitor input on other pages the user agent MUST set the {{XRSession}}'s [=visibility state=] to {{XRVisibilityState/"hidden"}} if the [=currently focused area=] does belong to the document which created the {{XRSession}}.


We should make it clear that this applies to both types of trusted UI

index.bs

Manishearth · 2019-10-21T22:27:25Z

Comments addressed.

toji

LGTM

NellWaliczek

A few more smallish pieces of feedback, but nothing that needs rereviewing once addressed. Thanks!

index.bs

Co-Authored-By: Nell Waliczek <[email protected]>

Manishearth · 2019-10-23T22:19:39Z

Addressed.

Manishearth · 2019-10-23T22:19:54Z

Thanks everyone!

…about immersive UIs

Fill out section on trusted UI

772daca

avadacatavra reviewed Oct 16, 2019

View reviewed changes

add the properties diane came up with

3bcdfe8

toji reviewed Oct 17, 2019

View reviewed changes

remove mention of desktop

effd538

NellWaliczek reviewed Oct 18, 2019

View reviewed changes

toji added this to the October 2019 milestone Oct 21, 2019

Manishearth changed the title ~~[WIP] Fill out section on trusted UI~~ Fill out section on trusted UI Oct 21, 2019

NellWaliczek assigned Manishearth Oct 21, 2019

Manishearth marked this pull request as ready for review October 21, 2019 22:27

Manishearth force-pushed the trusted-ui branch from 413b4e0 to ec4d15f Compare October 21, 2019 22:27

Address Nell's comments

992c094

Manishearth force-pushed the trusted-ui branch from ec4d15f to 992c094 Compare October 21, 2019 22:37

toji approved these changes Oct 21, 2019

View reviewed changes

This was referenced Oct 22, 2019

What is Trusted Immersive UI? #718

Closed

Is Trusted Immersive UI a conformance requirement? #719

Closed

NellWaliczek approved these changes Oct 23, 2019

View reviewed changes

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

Manishearth and others added 6 commits October 23, 2019 15:11

Apply suggestions from code review

ffaa000

Co-Authored-By: Nell Waliczek <[email protected]>

SHOULD -> MUST for reserved button

2e45ad9

remove sensitive

b36b143

Move paragraph

4a0b358

Move text about permissions prompts below

91a9c44

Clean up snooping

dd9065c

Manishearth merged commit 44735a2 into immersive-web:master Oct 23, 2019

Manishearth deleted the trusted-ui branch October 23, 2019 22:19

avadacatavra pushed a commit to avadacatavra/webxr that referenced this pull request Nov 14, 2019

summarized immersive-web#718, immersive-web#719, and immersive-web#875 …

b3255f8

…about immersive UIs

toji pushed a commit that referenced this pull request Nov 15, 2019

summarized #718, #719, and #875 about immersive UIs

18fec4b

kearwood pushed a commit to kearwood/webxr that referenced this pull request Mar 11, 2020

summarized immersive-web#718, immersive-web#719, and immersive-web#875 …

d07be48

…about immersive UIs

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fill out section on trusted UI #875

Fill out section on trusted UI #875

Manishearth commented Oct 14, 2019 •

edited

Loading

avadacatavra Oct 16, 2019

Manishearth Oct 16, 2019

Manishearth commented Oct 16, 2019 via email

toji Oct 17, 2019

Manishearth Oct 17, 2019

NellWaliczek left a comment

NellWaliczek Oct 18, 2019

NellWaliczek Oct 18, 2019

NellWaliczek Oct 18, 2019

NellWaliczek Oct 18, 2019

NellWaliczek Oct 18, 2019

NellWaliczek Oct 18, 2019

NellWaliczek Oct 18, 2019

Manishearth commented Oct 21, 2019

toji left a comment

NellWaliczek left a comment

Manishearth commented Oct 23, 2019

Manishearth commented Oct 23, 2019

		- Pausing the immersive session and showing some form of desktop environment in which a prompt can be shown

		</div>




		If the virtual environment does not consistently track the user's head motion with low latency and at a high frame rate the user may become disoriented or physically ill. Since it is impossible to force pages to produce consistently performant and correct content the user agent MUST provide a tracked, trusted environment and an [=XR Compositor=] which runs asynchronously from page content. The compositor is responsible for compositing the trusted and untrusted content. If content is not performant, does not submit frames, or terminates unexpectedly the user agent should be able to continue presenting a responsive, [=trusted UI=].


		While the API does not yet expose eye tracking capabilities a lot can be inferred about where the user is looking by tracking the orientation of their head. This is especially true of XR devices that have limited input capabilities, such as Google Cardboard, which frequently require users to control a "gaze cursor" with their head orientation. This means that it may be possible for a malicious page to infer what a user is typing on a virtual keyboard or how they are interacting with a virtual UI based solely on monitoring their head movements. For example: if not prevented from doing so a page could estimate what URL a user is entering into the user agent's URL bar.

		To prevent this risk the user agent MUST set the [=visibility state=] of all {{XRSession}}s to {{XRVisibilityState/"hidden"}} when the user is interacting with sensitive, trusted UI such as URL bars or system dialogs. Additionally, to prevent a malicious page from being able to monitor input on other pages the user agent MUST set the {{XRSession}}'s [=visibility state=] to {{XRVisibilityState/"hidden"}} if the [=currently focused area=] does belong to the document which created the {{XRSession}}.

Fill out section on trusted UI #875

Fill out section on trusted UI #875

Conversation

Manishearth commented Oct 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Manishearth commented Oct 16, 2019 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NellWaliczek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Manishearth commented Oct 21, 2019

toji left a comment

Choose a reason for hiding this comment

NellWaliczek left a comment

Choose a reason for hiding this comment

Manishearth commented Oct 23, 2019

Manishearth commented Oct 23, 2019

Manishearth commented Oct 14, 2019 •

edited

Loading