Issues importing the library in a Nuxt3/Vue3/Vite environment

[patratel]

Hi,

This might be a very basic issue that I'm simply unable to solve (probably due to lack of knowledge of how wasm works). I'm trying to import the modules into my Vue App in order to encode AVIF files and i'm getting the following error

500 Cannot find module 'C:\Users\cab_l\Desktop\Repositories\wsp-3.0\node_modules\@jsquash\avif\encode' imported from C:\Users\cab_l\Desktop\Repositories\wsp-3.0\node_modules\@jsquash\avif\index.js

I did check the node_modules folder and the module is present thus I have no idea why that might be the case.

My code looks like this

./composables/useUtiles.ts

import { encode } from "@jsquash/avif";

  const convertToAvif = (file) => {
    return new Promise(async (res, rej) => {
      console.log("file ", file);
      const avifBuffer = await encode(file);
      console.log("buffer ", avifBuffer);
      res();
    });
  };

I'm guessing that the issue might be related to this step in the readme file that I'm failing to understand how to proceed with

Note: You will need to either manually include the wasm files from the codec directory or use a bundler like WebPack or Rollup to include them in your app/server.

Any help with this would be greatly appreciated

[jamsinclair]

Thanks for reporting this @patratel.

I think this might be related to the vite bundler that Vue uses. If you follow the steps in this part of the README, does it solve the issue?

[CodeF53]

Adding onto this for my experience in Nuxt, I followed the steps you linked in the README.

nuxt.config.ts

// https://nuxt.com/docs/api/configuration/nuxt-config
export default defineNuxtConfig({
  ...
  vite: {
    optimizeDeps: {
      exclude: ['@jsquash/png', '@jsquash/oxipng'],
    },
  },
})

I followed the recommended formatting set in the docs, so I am pretty confident its doing something

But I am still getting this error in the vite console

ERROR  Failed to resolve import "../../.." from "node_modules/@jsquash/oxipng/codec/pkg-parallel/snippets/wasm-bindgen-rayon-3d2df09ebec17a22/src/workerHelpers.js?v=ff7385f7". Does the file exist?

And this error when starting my app

Cannot find module '/home/f53/proj/MinZip/node_modules/@jsquash/png/encode' imported from /home/f53/proj/MinZip/node_modules/@jsquash/png/index.js

[CodeF53]

Editing line 54 of workerHelpers.js

    // the main thread).
-  const pkg = await import('../../..');
+  const pkg = await import('../../../squoosh_oxipng');
   await pkg.default(data.module, data.memory);

Fixes Failed to resolve import "../../..", I suspect this is because you can't import a directory unless that directory contains an index.js to be automatically imported

[patratel]

Hey thank you for your answers guys, I did try what was written in the documentation to no avail.

I tried the following things:

1. Adding the following lines to the nuxt config file

./nuxt.config.ts

vite: { optimizeDeps: { exclude: ["@jsquash/avif"], }, },

2. Based on CodeF53's answer I tried importing the encode package directly as such

import encode from "@jsquash/avif/encode";
which resulted in a different import error which is the following:

Cannot find module 'C:\Users\cab_l\Desktop\Repositories\wsp-3.0\node_modules@jsquash\avif\meta' imported from C:\Users\cab_l\Desktop\Repositories\wsp-3.0\node_modules@jsquash\avif\encode.js

3. I also tried looking for a similar import error in the ./node_modules/@jsquash/avif/encode.js file and came across the following 2 imports which seem to import the files directly and not the directories as mentioned in the above comment.

const avifEncoder = await import('./codec/enc/avif_enc_mt'); const avifEncoder = await import('./codec/enc/avif_enc');

I'm still at a loss how to make this work on my app

[patratel]

Hello,

I've digged deeper into this issue and finally was able to resolve it, mainly thanks to @CodeF53.

The fix that did it for me was the following:

1. I imported the encode module directly

./composables/useUtils.ts
import encode from "@jsquash/avif/encode.js";

2. I changed the import lines and added a final .js at the end of the files.

./node_modules/@jsquash/avif/encode.js

import  {defaultOptions} from './meta.js';
import { initEmscriptenModule } from './utils.js';

Thank you for your time and tips guys!

[CodeF53]

Modifying the content of the source in node_modules isn't a proper solution for me. We still need a proper fix

[patratel]

I agree, i reopened the issue

[CodeF53]

I found a fix for the "Cannot find module" error in Nuxt:
nuxt.config.ts:

// https://nuxt.com/docs/api/configuration/nuxt-config
export default defineNuxtConfig({
  ...
  build: {
    transpile: [/@jsquash\/.*/],
  },
})

But:

• this doesn't fix the Could not resolve "../../.." from @jsquash/oxipng
• I haven't found an alternative for Vite

[jamsinclair]

Thanks for looking further into this @CodeF53 and @patratel.

It looks like Nuxt anticipates all third party modules to use the CommonJS module format. The modules provided by jSquash are all ESM. I'd prefer not to add CommonJS as it's a legacy format and not relevant to the browser intended context of these jSquash modules.

This means to use the jSquash modules with Nuxt you'll need to manually mark them as needing to be transpiled to CommonJS exports. (As discovered by @CodeF53).

I'd recommend setting the Nuxt config to

export default defineNuxtConfig({
  build: {
    transpile: ["@jsquash/avif", "@jsquash/png"],
  },
  vite: {
    optimizeDeps: {
      exclude: ["@jsquash/avif", "@jsquash/png"],
    },
  },
});

I have a minimal example of running jSquash modules with Nuxt at https://codesandbox.io/p/sandbox/hardcore-wildflower-zfh93n. You might find it useful @patratel.

[jamsinclair]

this doesn't fix the Could not resolve "../../.." from @jsquash/oxipng

@CodeF53 when you have time could you please provide a reproducible example of this with a shareable interface, such as CodeSandbox. That would be super helpful.

I've got my own example working ok with @jsquash/oxipng, Nuxt, Vite, Vue over at https://codesandbox.io/p/sandbox/jsquash-issue-19-oxipng-8kdc5h

[patratel]

I tried what you suggested @jamsinclair and it worked like a charm, thanks for the fix!

If I may dare I would like to ask you another question regarding an issue I encountered. I'm using the @jsquash/avif library in a Web Worker in order to run the conversion in the background. This works well now in development, the issue is that when I run "nuxt build" it stops due to an issue:

Unexpected early exit. This happens when Promises returned by plugins cannot resolve. Unfinished hook action(s) on exit: 22:36:03 (vite:worker) transform "C:/Users/cab_l/Desktop/Repositories/wsp-3.0/assets/workers/convertWorker.js?worker" (commonjs--resolver) resolveId "./codec/enc/avif_enc_mt" "C:/Users/cab_l/Desktop/Repositories/wsp-3.0/node_modules/@jsquash/avif/encode.js" (vite:worker-import-meta-url) transform "C:/Users/cab_l/Desktop/Repositories/wsp-3.0/node_modules/@jsquash/avif/codec/enc/avif_enc_mt.js"

This is how I'm importing the worker in my code

import convertWorker from "@/assets/workers/convertWorker?worker";

I've also tried importing the worker as such to no avail:

`
import convertWorker from "@/assets/workers/convertWorker?worker&url";

const convWorker = new Worker(convertWorker, { type: "module" });
`

My intuition tells me this might be due to Vite rather than the library itself. If you've run into anything similar before I would be grateful for any help/insight.

[jamsinclair]

@patratel yeah I think this is an issue with Vite itself. I think you've already found the related issue over at vitejs/vite#13367.

I will note that the avif package contains a worker, so you are indeed loading a worker inside worker which matches with the other user's problems in that issue. Given that the AVIF processing is already happening in a worker, do you need to use a worker then? (i.e. you don't need to use a worker in your own source code)

[patratel]

Indeed I did scour the internet trying to find a fix to this. I wasn't aware that there is a worker already running inside of it. I resorted to the worker solution because without it whenever the encoding of the avif is happening my application freezes. It might be due to how I'm implementing the library, this is my code:

const convertToAvifFile = (file, size, name) => {
    return new Promise(async (res, rej) => {
      try {
        const loadedImage = await loadImage(file, size);

        const avifBuffer = await encode(loadedImage);
        const base64String = "data:image/jpg;base64," + arrayBufferToBase64(avifBuffer);
        const avifBlob = await dataUrlToFile(base64String, name, "image/avif");

        res(avifBlob);
      } catch (e) {
        console.log("error", e);
      }
    });
  };

This was the solution that worked in dev and didn't make the app freeze but presented the above error:

const convertToAvifFile = (file, size, name) => {
    return new Promise(async (res, rej) => {
      try {

        const loadedImage = await loadImage(file, size);

        if (window.Worker) {
          const convWorker = convertWorker();

          convWorker.postMessage(loadedImage);
          convWorker.onmessage = async (e) => {
            const base64String = "data:image/jpg;base64," + arrayBufferToBase64(e.data);
            const avifBlob = await dataUrlToFile(base64String, name, "image/avif");
            res(avifBlob);
          };
        } else {
          const avifBuffer = await encode(loadedImage);
          const base64String = "data:image/jpg;base64," + arrayBufferToBase64(avifBuffer);
          const avifBlob = await dataUrlToFile(base64String, name, "image/avif");

          res(avifBlob);
          console.log("Your browser doesn't support web workers.");
        }
      } catch (e) {
        console.log("error", e);
      }
    });
  };

I'm pretty sure it's the encoding part since I tried isolating it.

I also tried to build the project without using a worker and I came across the same issue once more:

Unexpected early exit. This happens when Promises returned by plugins cannot resolve. Unfinished hook action(s) on exit:              15:36:09  
(commonjs--resolver) resolveId "./codec/enc/avif_enc_mt" "C:/Users/cab_l/Desktop/Repositories/wsp-3.0/node_modules/@jsquash/avif/encode.js"
(vite:worker-import-meta-url) transform "C:/Users/cab_l/Desktop/Repositories/wsp-3.0/node_modules/@jsquash/avif/codec/enc/avif_enc_mt.js"

Thank you for helping with this so far since I realize this is outside of the scope of this issue.

[jamsinclair]

I believe the original issue has been resolved so I'll close this issue.

I've added a commit to update the README with additional information to get these modules working with Nuxt (b152ee5).

@patratel I've had a brief play around and hit the same issue you are only when the production code is built. This is a vite/bundling problem and not related to this project. I wish you all the best with coming up with a solution 🤞

[CodeF53]

yeah I think this is an issue with Vite itself. I think you've already found the related issue over at vitejs/vite#13367.

I would add a note about this to the ### Issues with Vite and Vue build environments section of the readme, or open a github issue and leave it open until Vite fixes it.

Sure, this problem is technically documented, as it shows up in this issue, but this issue is closed, so it was the last place I thought to look when trying to figure out what I was doing wrong for my build step.

[CodeF53]

vitejs/vite#13367 can be pretty easily worked around by forcing single threaded behavior.

I propose adding a forceSingleThread flag to the init functions of problematic modules. It would default to false, so users don't have to think about adding it unless they need it.

avif/encode.ts

- export async function init(module?: WebAssembly.Module) {
+ export async function init(module?: WebAssembly.Module, forceSingleThread: boolean = false) {
-   if (!isRunningInCloudflareWorker() && await threads()) {
+   if (!isRunningInCloudflareWorker() && await threads() && !forceSingleThread) {
      const avifEncoder = await import('./codec/enc/avif_enc_mt');
      emscriptenModule = initEmscriptenModule(avifEncoder.default, module);
      return emscriptenModule;
    }
    const avifEncoder = await import('./codec/enc/avif_enc');
    emscriptenModule = initEmscriptenModule(avifEncoder.default, module);
    return emscriptenModule;
  }

oxipng/optimise.ts

- export function init(moduleOrPath?: InitInput): void {
+ export function init(moduleOrPath?: InitInput, forceSingleThread: boolean = false): void {
    if (!wasmReady) {
-     const hasHardwareConcurrency = globalThis.navigator?.hardwareConcurrency > 1;
+     const hasHardwareConcurrency = globalThis.navigator?.hardwareConcurrency > 1 && !forceSingleThread;
  
      wasmReady = hasHardwareConcurrency ? threads().then((hasThreads: boolean) =>
        hasThreads ? initMT(moduleOrPath) : initST(moduleOrPath),
      ) : initST(moduleOrPath);
    }
  }

I apologize if I screwed up typescript annotations, I am still learning typescript.

In testing this fixes my issues with @jSquash/OxiPNG, and doesn't cause any losses to performance, as I am multithreading calls to optimise within my application.

[patratel]

Thank you for suggesting this solution @CodeF53 , I did try applying it to the @jsquash/avif library I'm using, with no success. The same build error pops up. I did notice that the files I modified do differ a bit from the ones you mentioned. These are the changes i made

avif/encode.js

`
export async function init(module, forceSingleThread = false) {

if (!isRunningInCloudflareWorker() && await threads() && !forceSingleThread) {

    const avifEncoder = await import('./codec/enc/avif_enc_mt');

    emscriptenModule = initEmscriptenModule(avifEncoder.default, module);

    return emscriptenModule;

}

const avifEncoder = await import('./codec/enc/avif_enc');

emscriptenModule = initEmscriptenModule(avifEncoder.default, module);

return emscriptenModule;

}`

avif/encode.d.ts

export declare function init(module?: WebAssembly.Module, forceSingleThread: boolean = false): Promise<AVIFModule>;

I also tried setting the flag to true to no avail, am I doing something wrong?

[CodeF53]

Try completely commenting out the Multithreaded portion of the init code, leaving only the singlethreaded option.

[patratel]

Wow! I can't thank you enough @CodeF53, commenting the first section worked! I can finally ditch the 'sharp' server workaround I've been using for the past 3 months. I do feel a bit like a monkey since I've no clue what this change did to make it work. If you ever do have the time and patience to give a short explanation as to why this workaround works and what could be the drawbacks.

Leaving my changes down here for anyone stumbling upon this issue in the future:

avif/encode.js
`
export async function init(module, forceSingleThread = true) {

// if (!isRunningInCloudflareWorker() && await threads() && !forceSingleThread) {

//     const avifEncoder = await import('./codec/enc/avif_enc_mt');

//     emscriptenModule = initEmscriptenModule(avifEncoder.default, module);

//     return emscriptenModule;

// }

const avifEncoder = await import('./codec/enc/avif_enc');

emscriptenModule = initEmscriptenModule(avifEncoder.default, module);

return emscriptenModule;

}`

./nuxt.config.ts

vite: { optimizeDeps: { exclude: ["@jsquash/avif"], }, worker: { format: "es", }, },

Adding the worker format in the nuxt config file was also required in order to circumvent another error.

Thanks again!

[jamsinclair]

Thanks @CodeF53. I had thought of a similar work around. Feel free to submit a PR and I can help add that code into the repository.

I've added a note to the README about this issue in commit bf6161a

[jamsinclair]

Edit: I had posted a hacky solution that does not work. I've removed it as it did not solve the problem. The real fix will need to happen in Vite.

[jamsinclair]

I've updated the docs with a workaround of special single thread only builds that don't use workers. This will bypass the Vite problem until it's solved.

See: https://github.com/jamsinclair/jSquash#issues-with-nuxtvite-and-nested-web-workers

[jamsinclair]

Thanks to all your efforts, I think we played a part in this getting fixed in Vite! 🎉

Vite's next Major release will contain the fix made in this PR (vitejs/vite#14685)

You can try this out now in the latest v5 beta versions. You can install with npm install -S vite@beta

Edit: Unfortunately, there is another Vite bug that still breaks the Vite build 😢 – vitejs/vite#7015