[feature request] support for web workers

[catdad]

Rendering PSD files can be quite slow and the current implementation in this module is synchronous and blocking. I'd like to hand off this work to a web worker, but that is currently not possible due to the use of canvas in the module.

I currently have 2 ideas for this:

• use OffscreenCanvas where available. Unfortunately, "where available" is only Chromium browsers, so that's certainly not ideal.
• best I can tell (I read some of this code for maybe 20 minutes, but I could have missed something), the canvas is used to insert data into using ImageData, and then exposed as the public API result so that it can be exported by the user using toDataURL(), toBlob(), or toBuffer() in node. However, ImageData is just a wrapper around Uint8ClampedArray, which can be used throughout the code instead of ImageData. The API could then be updated to specify whether to provide a canvas or to provide this Uint8ClampedArray along with the width and height directly. This array can be transferred across the worker and rendered on the other side (either by copying it into a new ImageData object or through various other means like jimp or sharp -- this is all up to the user, of course) doing only a small trivial portion in the main thread. I think this is the preferred option, but obviously quite a bit more work.

I am happy to help with all this, but I am hoping to get your input before starting.

[Agamnentzar]

There are 2 ways to handle this without any changes to the library:

1. With OffscreenCanvas it should be pretty easy , as you can just use initializeCanvas method and set your own implementation for createCanvas and createCanvasFromData, you can see that being used in initializeCanvas.ts to setup node-canvas handlers.

2. For writing easiest way would be to create fake Canvas objects with getContext method that just returns object with getImageData method that returns ImageData that you get from the canvas before sending data to the worker. I'm using webgl textures directly for saving PSD. The problem would be thumbnails, because they rely on drawImage method from canvas element so you'd only be able to create thumbnails on browsers supporting OffscreenCanvas it's a mess.
   Can do similar for reading, though you'd need to implement more fake methods like createImageData and putImageData

Otherwise, the best way to implement this into the library in my opinion would be to allow image field to be used instead of canvas field on layers and psd object that would contain the same fields as ImageData (width, height, buffer). Writer would just see image fields and use them instead of canvas field and for reader there would have to be an option to read into image field instead of canvas. Still wouldn't be able to avoid thumbnail issue this way though without implementing image scaling and JPEG encoding.

[catdad]

Okay, so it took me a minute to figure out what you meant -- I was stuck thinking about what all this meant about updating the library itself. But what you are actually saying is that I can myself use initializeCanvas as an end-user to implement my own canvas version. This means that no updates to the library are actually necessary -- clever.

So in my case (since it's an Electron app and I know exactly what browser will be used), I added this to my app (for anyone else looking for this):

const { readPsd, initializeCanvas } = require('ag-psd');
const base64 = require('base64-js');

const createCanvas = function (width, height) {
  console.log('MY CREATE CANVAS');

  const canvas = new OffscreenCanvas(width, height);
  canvas.width = width;
  canvas.height = height;
  return canvas;
};
const createCanvasFromData = function (data) {
  console.log('MY CREATE CANVAS FROM DATA');

  const image = new Image();
  image.src = 'data:image/jpeg;base64,' + base64.fromByteArray(data);
  const canvas = new OffscreenCanvas(image.width, image.height);
  canvas.width = image.width;
  canvas.height = image.height;
  canvas.getContext('2d').drawImage(image, 0, 0);
  return canvas;
};

Then I can use it like this:

// in Web Worker
const psd = readPsd(myInput, {
  skipLayerImageData: true
}));
const canvas = psd.canvas;

// OffscreenCanvas doesn't have the same image export methods,
// but you can get a buffer, which is frankly the best one
const blob = await canvas.convertToBlob({ type: 'image/jpeg', quality: 0.92 });
const arrayBuffer = await blob.arrayBuffer();

// ArrayBuffer is a transferable!! Send that to the main thread
postMessage({ data: arrayBuffer, width: canvas.width, height: canvas.height }, [arrayBuffer]);

// in the main thread (Electron with node integration)
const buffer = Buffer.from(message.data);

// in the main thread (in a browser)
const canvas = document.createElement('canvas');
canvas.width = message.width;
canvas.height = message.height;
const ctx = canvas.getContext('2d');
const imageData = ctx.createImageData(message.width, message.height);

for (let i = 0; i < message.data.length; i++) {
  imageData.data[i] = data[i];
}

ctx.putImageData(imageData, 0, 0);

// now you have a canvas with the image in it, do whatever you wanted to do before
const dataUrl = canvas.toDataURL('image/jpeg', 0.92);

So it turns out my use case is actually covered already. For browsers that do not have OffscreenCanvas, I did manage to find a synchronous jpeg encoder -- jpeg-js -- of course, that is assuming the user only wants jpeg. The more popular options (which include all sorts of export formats) are all asynchronous, for good reasons. But you are right, it would be quite messy. It might be better to leave for someone adventurous who wants to implement this for non-Chromium browsers?

For writing, I am checking out what is being used, and it seems like only drawImage is used for scaling the image, and everything else is based on ImageData. Is that right? That one would be much tougher, and I can't find an existing implementation that is synchronous, so a user probably wouldn't be able to fill this in themselves.

[Agamnentzar]

it's drawImage for scaling and then getDataURL for encoding to jpeg, also new Image and image.src for decoding, it would be a lot of work just for thumbnails

btw I think you can just send offscreen canvas to and from the worker and then draw it directly to regular one when you have it back

[catdad]

Looks like I made a few mistakes in that code above (that's what I get for not running it first). I'll fix it up once I clean up my implementation a bit, in case anyone wants to use it in the future.

Also, unfortunately, you cannot send an OffscreenCanvas. It is not a serializable object, so you get this error:

Failed to execute 'postMessage' on 'DedicatedWorkerGlobalScope': An OffscreenCanvas could not be cloned because it was not transferred.: DataCloneError

And in case you interpret that message as just needing to transfer it, it is also not a transferable:

Failed to execute 'postMessage' on 'DedicatedWorkerGlobalScope': An OffscreenCanvas could not be cloned because it had a rendering context.: DataCloneError

[Agamnentzar]

Did you mark it when sending over, you should be able to do it according to this: https://developers.google.com/web/updates/2018/08/offscreen-canvas#unblock_main_thread

That feature might not be in electron yet though, they just added it to chrome

[catdad]

Ooh, interesting. I hadn't read this post, I was just going off of what MDN said. It looks like there are a few caveats to using it this way though. I will look into it now.

[Agamnentzar]

@Agamnentzar add test/example of using ag-psd in a webworker

[Agamnentzar]

Closed in 25ee647