Support for delayed clipboard data

[poiru]

Apple explains this well:

If a pasteboard item supports multiple representations, it is typically impractical or time- and resource-consuming to put the data for each representation onto the pasteboard. For example, say your application places an image on the pasteboard. For maximum compatibility, it might be useful for the image to provide a number of different representations, including PNG, JPEG, TIFF, GIF, and so on. Creating each of these representations, however, would require time and memory.

Rather than requiring an item to provide all of the representations it offers, a pasteboard only asks for the first representation in the list of representations an item offers. If a paste recipient wants a different representation, the item can generate it when it’s requested.

Has something like this been considered? For example, instead of:

var data = new DataTransfer();
data.items.add("text/plain", "foo");
data.items.add("expensive", generateExpensiveData());
navigator.clipboard.write(data);

we could have something like:

var data = new DataTransfer();
data.items.add("text/plain", "foo");
data.items.add("expensive", function () {
  // Called only when something tries to read data of the "expensive" type
  return generateExpensiveData();
});
navigator.clipboard.write(data);

[lgarron]

For maximum compatibility, it might be useful for the image to provide a number of different representations, including PNG, JPEG, TIFF, GIF, and so on.

This doesn't necessarily mitigate the desire to account for it, but note that Chrome is unlikely to support copying images without browser-side re-encoding for sanitization (that's one major motivation for the asynchronous API). So it wouldn't be beneficial to generate various image formats before passing to the API in that case.

[lgarron]

data.items.add("expensive", function () {
// Called only when something tries to read data of the "expensive" type
return generateExpensiveData();
});

Presumably most OSes require pushing the data actively to the clipboard. Would it make sense to use a Promise here, since we are directly triggering the write with the API?

[poiru]

This doesn't necessarily mitigate the desire to account for it, but note that Chrome is unlikely to support copying images without browser-side re-encoding for sanitization (that's one major motivation for the asynchronous API). So it wouldn't be beneficial to generate various image formats before passing to the API in that case.

That's fine for our needs, at least. We need this for non-image data (e.g. PDF for interop with Adobe products).

Presumably most OSes require pushing the data actively to the clipboard. Would it make sense to use a Promise here, since we are directly triggering the write with the API?

All major desktop platforms (i.e. Windows, macOS, and Linux) have a way to add format(s) to the clipboard and provide the data only when requested.

A promise does not solve the problem we're trying to address. In our case, we are developing a design tool that ideally would interoperate with e.g. Adobe Illustrator, Sketch, Glyphs, etc. To maximize the number of interoperable applications, we need to be able to place arbitrary formats on the clipboard (e.g. com.adobe.pdf, image/svg+xml, etc.) without generating the data for all of those formats. If we were to do everything on copy, our app would appear frozen/unresponsive while it generates all that data. Generating a single format only on paste would be a much better user experience.

[lgarron]

All major desktop platforms (i.e. Windows, macOS, and Linux) have a way to add format(s) to the clipboard and provide the data only when requested.

Ooh, cool, I did not know.

That makes me think of Chrome Remote Desktop, which could make use of a delayed clipboard the other way. Right now, I believe reads the clipboard every time you switch the app, and pushes that to the remote computer. If it had a clipboard listener whose handler could asynchronously query for formats it could also benefit from those OS mechanisms, right?

[poiru]

That makes me think of Chrome Remote Desktop, which could make use of a delayed clipboard the other way. Right now, I believe reads the clipboard every time you switch the app, and pushes that to the remote computer. If it had a clipboard listener whose handler could asynchronously query for formats it could also benefit from those OS mechanisms, right?

The remote Chrome instance would have to block whatever thread processes the clipboard data request until it receives the data from the local Chrome instance. It is probably doable, but network latency could be a problem.

[gked]

@poiru :

All major desktop platforms (i.e. Windows, macOS, and Linux) have a way to add format(s) to the clipboard and provide the data only when requested.

Could you please clarify this?
Assuming I understood the scenario correctly, I am not aware of a way to tell the clipboard to go back and get the format needed from the origin on Windows.

[poiru]

Assuming I understood the scenario correctly, I am not aware of a way to tell the clipboard to go back and get the format needed from the origin on Windows.

I linked to this earlier. Let me know if that page is not helpful!

[gked]

Thanks, taking a look.

I've also noticed in your code example, you were using APIs from the modern clipboard spec navigator.clipboard.write(data);. Did you mean to post the issue there?

[garykac]

@gked : That was the incubation document. We've folded the Async Clipboard API info into the main clipboard spec, so this seems like the best place to track the issue.

@poiru : Do you know how that API works with multiple apps writing to the clipboard? E.g.:

• App1 writes delayed info into clipboard
• App2 reads general clipboard data
• App1 quits
• App2 tries to read the "delayed" data from the clipboard

From a quick glance, I couldn't convince myself that it was designed to work outside of a single app managing its own clipboard.

[poiru]

@garykac Applications that own the clipboard with delayed data usually warn on quit. For example, Adobe Illustrator shows this:

Keep Data generates data for all formats and adds them to the clipboard so that they can outlive the application.

[alecazam]

The image sanitization is actually not great for our design tool. Users need the metadata and original content, so I'd recommended an early-out off sanitization detection first. If the image doesn't need to be sanitized, then pass through the original. For example, animated PNG might be converted to a regular PNG by the sanitization.

[CendioOssman]

I see that remote chrome is already mentioned here, but I'd like to add to that that there are many other remote desktop protocols (RDP, VNC, Spice, etc.) that could all benefit from not having to transfer the data whenever the clipboard changes.

The user will in most cases not even use the clipboard so there will be a lot of wasted bandwidth. Bandwidth that could have been used for transferring image updates instead.

It would be nice if this could be added before the async API is finalised.

(This also affects DragEvent and ClipboardEvent, although it's probably rare that their inefficiency is a problem)

[garykac]

Proposal: https://docs.google.com/document/d/1sMcajoYAQ_M_VOIn1RLokG1sVB7skjoeviqt9aZOcDk/edit