Don't recommend user gesture in clipboard API

[CendioOssman]

The new async clipboard API suggestion is a very nice step forward. However it still (vaguely) recommends user gestures as a way to prevent abuse. This method makes the clipboard API completely unusable for our use case so I'd like to plead for a recommendation against that in favour of other protection.

The use case here are remote desktop applications. The entire interface is server side so the Javascript is not aware of any of the logic, or even the layout. Hence it cannot connect user gestures with a clipboard update.

The flow would basically be:

1. The user presses Ctrl+C, resulting in a series of keydown events.
2. The page forwards these events to the remote system, exiting the event handler
3. The remote application triggers a Copy operation based on these events and sends some clipboard data to the client
4. The page receives the data and tries to update the local clipboard

This method is currently blocked, and would continue to be blocked unless browsers stop requiring a user gesture to access the API.

[mbrodesser]

@CendioOssman: does it imply that all JS functionality requiring transient activation is broken when using a remote desktop application like Cendio?

If there's are any news about this issue, please share them here.

[CendioOssman]

Anything supporting transient activation should work fine in our scenarios, if I'm reading that description correctly¹. Assuming the transient activation duration is long enough to cover sending the notification over the network, having the remote applications react, and send a request back.

What won't work is things that must be done before returning from an event handler. We are simply too asynchronous for that. And it was my understanding that this was/is how some user activated things are blocked?

¹ There might be some corner cases, but I would assume 99% of applications only do clipboard operations as a result of a keyboard key or mouse button

[mbrodesser]

Thanks for your fast response.

Anything supporting transient activation should work fine in our scenarios, if I'm reading that description correctly¹. Assuming the transient activation duration is long enough to cover sending the notification over the network, having the remote applications react, and send a request back.

What won't work is things that must be done before returning from an event handler. We are simply too asynchronous for that. And it was my understanding that this was/is how some user activated things are blocked?

Yes, Safari apparently requires calling clipboard.readText() from an event handler, which corresponds to the user gesture (see "Security and Privacy" at https://webkit.org/blog/10855/).

For Gecko/Firefox, I'm currently considering to allow calling clipboard.readText() outside of such event handlers, limited to the same frame (or same origin) and as long as it's called within the transient activation duration.
If I understand correctly, that should cover your use-case.

But I'm wondering, couldn't your application use an async event handler and wait for the result in there?

¹ There might be some corner cases, but I would assume 99% of applications only do clipboard operations as a result of a keyboard key or mouse button

[mbrodesser]

But I'm wondering, couldn't your application use an async event handler and wait for the result in there?

Partially answering my own question: with Safari one might run into the limitation mentioned in https://bugs.webkit.org/show_bug.cgi?id=222262.

However, from a security and privacy perspective, I don't understand that limitation.
@whsieh, can you please shed some light on that?

[snianu]

Adding Agenda label since this might affect the permission PR #164 as well.

[CendioOssman]

But I'm wondering, couldn't your application use an async event handler and wait for the result in there

Weak "maybe" on that. The problem is that we don't know which events will trigger a clipboard request. Which means we would need to assume every event could do so. So how would we do a wait if we don't know if that wait would ever complete?

Might be some method that works here, but it sounds very hacky and brittle.

[CendioOssman]

Hmm... Transient activation might be an issue on clipboard write. Transferring the clipboard data could be a very slow process for large things (e.g. huge image, or a file).

Does the API consider this scenario? I.e. we would need some way to signal that we will be writing to the clipboard, we just need more time to actually provide types and contents.

[snianu]

ClipboardItemData takes a Promise to the Blob/DOMString that you can resolve whenever the data is available. The transient user activation restriction is only for the clipboard.write() call.

[CendioOssman]

I think that should work for us the way things look right now. There is one gotcha in that the mime types need to be provided up front, not via a Promise. So for protocols that don't put the types early this might be an issue. Hopefully those are rare.

[mbrodesser]

I think that should work for us the way things look right now.

@CendioOssman: does it mean this ticket should be closed?

There is one gotcha in that the mime types need to be provided up front, not via a Promise. So for protocols that don't put the types early this might be an issue. Hopefully those are rare.

True, interesting corner case.

[whsieh]

But I'm wondering, couldn't your application use an async event handler and wait for the result in there?

Partially answering my own question: with Safari one might run into the limitation mentioned in https://bugs.webkit.org/show_bug.cgi?id=222262.

However, from a security and privacy perspective, I don't understand that limitation. @whsieh, can you please shed some light on that?

The immediate explanation for this limitation is that we (WebKit) use the same UserGestureIndicator check here that we use in other places that require user activation (such as showing UI for file pickers). This means that the user gesture indicator should be propagated through setTimeout and fetch requests, for up to 1 second, but otherwise only persists during the scope of handling a user activation event (e.g. click or touchstart/touchend).

I think we could probably relax this constraint, such that the "user interaction" is valid as long as:

1. We handled a user activation event recently (probably, at most ~1 sec. ago?).
2. The browser tab hasn't been backgrounded.
3. Nothing else has attempted to write to the clipboard in the meantime.

[annevk]

@whsieh that's great. It would be ideal if it was aligned with the transient activation model of the HTML Standard. The more features rely upon the same mechanism the better for web developers.

[CendioOssman]

@CendioOssman: does it mean this ticket should be closed?

I'm not familiar with your procedures, but if everything here uses transient activation then we should be good for our use cases. Safari needs to be brought in line though before things will work in practice.

1. We handled a user activation event recently (probably, at most ~1 sec. ago?).

I would prefer a few seconds to handle performance hiccups (CPU, network, etc.) that could cause delays. We have more than just JavaScript performance in the browser to worry about.

It would also be nice if there is some consensus among the browser implementations on this point.

[mbrodesser]

The immediate explanation for this limitation is that we (WebKit) use the same UserGestureIndicator check here that we use in other places that require user activation (such as showing UI for file pickers). This means that the user gesture indicator should be propagated through setTimeout and fetch requests, for up to 1 second, but otherwise only persists during the scope of handling a user activation event (e.g. click or touchstart/touchend).

@whsieh thanks for the explanation.

I think we could probably relax this constraint, such that the "user interaction" is valid as long as:

1. We handled a user activation event recently (probably, at most ~1 sec. ago?).

In Gecko/Firefox it's currently 5 seconds (https://searchfox.org/mozilla-central/rev/cf77e656ef36453e154bd45a38eea08b13d6a53e/modules/libpref/init/StaticPrefList.yaml#3775-3778).

2. The browser tab hasn't been backgrounded.

3. Nothing else has attempted to write to the clipboard in the meantime.

The third point refers to clipboard.write() and clipboard.writeText() only and not to read() and readText(), correct?
With "Nothing else", do you mean no other script? Presumably native applications could always write (already now in Safari).
If this was indeed only about write() and writeText() I wonder if 2. and 3. are indeed necessary.

[css-meeting-bot]

The Web Editing Working Group just discussed Don't recommend user gesture in clipboard API.

The full IRC log of that discussion

<Travis> topic: Don't recommend user gesture in clipboard API
<Travis> github: https://github.com//issues/75
<Travis> johanneswilm: last time the action was whsieh will check user-activation spec and propose number of seconds.
<Travis> .. and BoCupp said 1s seems reasonable.
<Travis> whsieh: I haven't yet decided on a specific value of seconds... :(
<Travis> .. I think we should just start with 1s and increase if needed.
<Travis> BoCupp: refresher: why the number of seconds? (Can't remember the details)
<Travis> .. when writing to clipboard you can write immediately. Waiting on the network, etc., isn't a problem except for the timing of performing the write call.
<Travis> whsieh: in this case the requestor doesn't know whether they want to call clipboard write during the user gesture.
<Travis> johanneswilm: e.g., in remote desktop they don't know if anything's on the clipboard.
<Travis> .. do I hear correctly that 1s is the recommendation?
<Travis> whsieh: I think we should start more conservatively. Chrome/Firefox time is currently 5s.
<Travis> .. Webkit's user activation time is 1s everywhere else. This would match everything else.
<Travis> .. then eventually we could move everything to match other browser (user-activation time delay to perhaps 5s)
<Travis> .. right now there is no delay--it has to be during the event handler.
<Travis> BoCupp: Even in navigator.clipboard.write?
<Travis> whsieh: yes, in webkit.
<Travis> .. like touchstart--other things we consider user activation.
<Travis> .. it has to be synchronously as part of the user gesture (no async stuff)
<Travis> johanneswilm: So, 1s? Will other browsers change?
<Travis> BoCupp: I feel like (no number of seconds) will ever be enough for some of these scenarios.
<Travis> johanneswilm: should there be a minimum?
<Travis> .. e.g., in firefox it's 5s? But other browsers have faster times (500ms), then it will break everywhere else.
<Travis> .. but if it's written down as 1s, then there's an expectation set.
<Travis> BoCupp: I'm not inclinded to write the 1s (or other time) anywhere. Would rather provide affordances for delaying rather than just changing the number of seconds for this [edge case].
<Travis> .. 5s seems like a really long time for a user-gesture...
<Travis> BoCupp: Would like to say in the issue--not likely to change this. There's no guarantee that this will solve all use cases...
<Travis> .. wondering if this is the kind of issue we don't need to put in the spec.
<Travis> whsieh: having some delay (in WebKit) would help, since there's currently no delay.
<Travis> johanneswilm: So, we don't want to specify any value, but should just say some amount of time?
<Travis> BoCupp: If we did say it, we wouldn't put it in our spec... should probably go to user-activation spec?
<Travis> .. increasing the seconds to solve this problem isn't really solving the problem.
<Travis> .. I do like having a consideration for introducing an async delay.
<Travis> .. I like the idea of having whsieh post to the issue and indicate they may take some action...
<Travis> action: whsieh to post to the issue and indicate a plan to make an adjustment to support async delay of some sort to handle this scenario.

[whsieh]

We (WebKit) will experiment with allowing clipboard writes to be initiated with a timeout of 1 second, which I believe should mostly address the primary issue here (which is that there is currently no "activation grace period" in WebKit). 1 second is also consistent with the existing maximum user gesture forwarding time limit in WebKit.

I think we can discuss relaxing this to 5 seconds to match Firefox in the future.