Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Screen Enumeration API #413

Closed
3 of 5 tasks
bunneyster opened this issue Sep 3, 2019 · 13 comments
Closed
3 of 5 tasks

Screen Enumeration API #413

bunneyster opened this issue Sep 3, 2019 · 13 comments
Assignees
Labels
Provenance: Fugu Review type: CG early review An early review of general direction from a Community Group Topic: privacy

Comments

@bunneyster
Copy link

こんにちはTAG!

I'm requesting a TAG review of:

Further details:

We recommend the explainer to be in Markdown. On top of the usual information expected in the explainer, it is strongly recommended to add:

  • Links to major pieces of multi-stakeholder review or discussion of this specification:
  • Links to major unresolved issues or opposition with this specification:

You should also know that...

This API is intended to be used in conjunction with the Window Placement API, which is still in the proposal stage.

We'd prefer the TAG provide feedback as (please select one):

  • open issues in our GitHub repo for each point of feedback
  • open a single issue in our GitHub repo for the entire review
  • leave review feedback as a comment in this issue and @-notify [github usernames]

Please preview the issue and check that the links work before submitting. In particular, if anything links to a URL which requires authentication (e.g. Google document), please make sure anyone with the link can access the document.

¹ For background, see our explanation of how to write a good explainer.

@torgo
Copy link
Member

torgo commented Sep 10, 2019

Discussing in the f2f right now - @lknik can you take a look at the answers to the privacy & security questionnaire?

@hober
Copy link
Contributor

hober commented Oct 15, 2019

Labeling as a Fugu-related request since it appears in Fugu's full list of capabilities.

@lknik lknik self-assigned this Oct 17, 2019
@lknik
Copy link
Member

lknik commented Oct 18, 2019

Each of the new display properties chosen for this API makes a non-replaceable contribution. Removal or constraining of any property may degrade the user experience. Some properties offer more value than others, and some are more relevant to the proposed use cases than others. Care should be taken to choose those properties offering the most value to the most important use cases.

It has been suggested that user agents themselves could provide a UI for selecting where content is presented, but this is cumbersome for users and prohibitively limiting for web applications. This also offers no meaningful value over the existing cumbersome manual placement of windows by users.

Can't we simplify to "for each display we need same information as is currently offered by existing screen object"? We do not discuss the contents of the screen object, but the inflation of (many) screen objects.

2.3 This API does not expose such information. Any exposed information may identify the machine, but not its user.

Information allowing to identify the user's machine is more or less identifying the user, so you're saying the opposite. Maybe at least remove the 2nd sentence?

2.6 This API proposes exposing some additional properties for each of the displays connected to the device; see Section 2.1 for a complete list.

The complete list seems to be X*12-15 static objects (some of them dynamic; with X the number of screens).

2.7 Does this specification allow an origin access to sensors on a user’s device? No.

It does on the other hand inform if the display has an accelerometer, which would make it simpler to direct a malicious script to an appropriate window.

2.8 "The following display properties are currently exposed ..."

So we can today access all the device screens already? Wouldn't it be more useful to enumerate the new identifiers to be exposed, which is the above 12-15 numbers per each screen, which is not currently easily available?

To mitigate this issue, user permission is required to access the display

Do I see it correctly that this API will be gated behind permissions?

@lknik lknik added the Review type: CG early review An early review of general direction from a Community Group label Oct 18, 2019
@hober hober added Progress: pending external feedback The TAG is waiting on response to comments/questions asked by the TAG during the review and removed Progress: unreviewed labels Dec 3, 2019
@hober
Copy link
Contributor

hober commented Dec 3, 2019

Marking as pending external feedback; @lknik‘s comments above have not been addressed, and we’ve also raised webscreens/screen-enumeration#23.

@plinss
Copy link
Member

plinss commented Mar 2, 2020

Looked again during Wellington F2F, still waiting for external feedback. We also have concerns about how this intersects with ongoing work for foldable screens #472.

An additional observation is that there's an explicit non-goal of exposing all the OS screen APIs yet this proposal seems to do so.

@hober hober added this to the 2020-05-21-f2f-seoul milestone May 7, 2020
@hober
Copy link
Contributor

hober commented May 27, 2020

@sPark008 I'm not sure if you saw @lknik's questions above. Could you take a look at them and get back to us?

@michaelwasserman
Copy link

Thank you for poking this thread and apologies for the delay!
I'll try to address the open questions and feedback this week.

@michaelwasserman
Copy link

michaelwasserman commented May 29, 2020

Thank you for your feedback and patience! I hope you find my responses and changes helpful, and I look forward to iterating further with additional input.

@hober thanks for your feedback and patience, I hope you find these responses helpful:

We also have concerns about how this intersects with ongoing work for foldable screens #472.

This work is complimentary to Foldable Screens #472. That proposal offers simple tools to support content spanning the single fold of a dual-screen device. Screen Enumeration and Window Placement (TAG review request imminent) aims to support a broader range of multi-screen and multi-window scenarios with lower-level information (eg. 3+ screens, 2 heterogeneous or misaligned screens, separate windows on separate screens, etc.).

An additional observation is that there's an explicit non-goal of exposing all the OS screen APIs yet this proposal seems to do so.

The goal is to expose a reasonable set of info for web applications to make informed use of the available screen space; that will require exposing some limited new information, but care has been taken to select the most valuable subset of available information. The spirit of the rephrased non-goal is to avoid exposing extraneous or especially identifying information (eg. EDID), and avoid exposing APIs to control the configuration of display devices.

we’ve also raised webscreens/screen-enumeration#23.

I posted a couple comments there, and want to explore ways to support requests for limited screen information. Please let me know if you have any thoughts regarding my comments there.

@lknik Thank you for your Security and Privacy review and comments, I hope you find the latest changes there and these responses helpful:

Each of the new display properties chosen for this API makes a non-replaceable contribution. Removal or constraining of any property may degrade the user experience. Some properties offer more value than others, and some are more relevant to the proposed use cases than others. Care should be taken to choose those properties offering the most value to the most important use cases.

It has been suggested that user agents themselves could provide a UI for selecting where content is presented, but this is cumbersome for users and prohibitively limiting for web applications. This also offers no meaningful value over the existing cumbersome manual placement of windows by users.

Can't we simplify to "for each display we need same information as is currently offered by existing screen object"? We do not discuss the contents of the screen object, but the inflation of (many) screen objects.

I modified the section to separate the goals of exposing multiple screens and new screen properties , both of which are valuable.

2.3 This API does not expose such information. Any exposed information may identify the machine, but not its user.

Information allowing to identify the user's machine is more or less identifying the user, so you're saying the opposite. Maybe at least remove the 2nd sentence?

You're right, I revised the section to accurately convey that new information is exposed that could be used to help identify a device, and mention the possibility to gate access upon user permission.

2.6 This API proposes exposing some additional properties for each of the displays connected to the device; see Section 2.1 for a complete list.

The complete list seems to be X*12-15 static objects (some of them dynamic; with X the number of screens).

I added a comparable summary in this section and added screen.[avail]top/left in section 2.1.

2.7 Does this specification allow an origin access to sensors on a user’s device? No.

It does on the other hand inform if the display has an accelerometer, which would make it simpler to direct a malicious script to an appropriate window.

I added a note to this effect in the section.

2.8 "The following display properties are currently exposed ..."

So we can today access all the device screens already? Wouldn't it be more useful to enumerate the new identifiers to be exposed, which is the above 12-15 numbers per each screen, which is not currently easily available?

I rewrote this section to give a more pointed answer to its question, including the difference of the single screen available to each window and the full set of screens, and a summary of what's currently available for each considered/proposed property.

To mitigate this issue, user permission is required to access the display

Do I see it correctly that this API will be gated behind permissions?

Yes, user permission should be a critical consideration of any browser that might adopt something like this proposal, and the asynchronous getScreens() should enable user-facing permission prompts.

@michaelwasserman
Copy link

Please consider #522 alongside this proposal, they're highly related; thank you!

@kenchris kenchris self-assigned this Jun 10, 2020
@kenchris kenchris removed the Progress: pending external feedback The TAG is waiting on response to comments/questions asked by the TAG during the review label Jun 10, 2020
@michaelwasserman
Copy link

You may wish to consider these specific questions during review. Thank you!

@hober
Copy link
Contributor

hober commented Jul 27, 2020

We talked about this during a breakout today. We'll leave comments on each of these issues.

@michaelwasserman
Copy link

I believe I have addressed all recent feedback, thank you!

@plinss plinss removed the Progress: pending external feedback The TAG is waiting on response to comments/questions asked by the TAG during the review label Aug 24, 2020
@kenchris
Copy link

Hi there,

We are fine with the work you have done and we are therefore going to close this issue. @plinss is still a bit worried that part of this might ship in a way that it is incompatible with the work around foldables, and will file an issue about that on your new repo.

Thanks for staying at home with the TAG!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Provenance: Fugu Review type: CG early review An early review of general direction from a Community Group Topic: privacy
Projects
None yet
Development

No branches or pull requests

7 participants