webkitRequestFileSystem and brotli_wasm_bg.wasm problem

[szanuje]

Thanks for reporting an issue!

Steps to reproduce

Add package to NextJS 14 project

Expected behaviour

Package imported correctly

Actual behaviour

Package is throwing error on NextJS build

Any logs, error output, etc?

Error is either:

Error: ENOENT: no such file or directory, open '/Users/<projectPath>/.next/server/chunks/brotli_wasm_bg.wasm'

or

 Collecting page data  ...TypeError: Cannot read properties of undefined (reading 'webkitRequestFileSystem')

depending on which import statement I use:

import parquetjs from "@dsnp/parquetjs";
import parquetjs from "@dsnp/parquetjs/dist/browser/parquet.esm";

Any other comments?

...

[wilwade]

@szanuje Which version of webpack are you using? We've had a few issues with correctly handling webpack that might be related:

• Webpack Issues #116
• Module util could not be found #120

[JasonYeMSFT]

'brotli-wasm' requires the wasm binary in a way that is very unfriendly to web bundlers.

const path = require('path').join(__dirname, 'brotli_wasm_bg.wasm');
const bytes = require('fs').readFileSync(path);

The dynamic fs read prevents the bundler from bundling the wasm, although we implemented a custom wasm loader.
The root cause seems to be this rustwasm/wasm-pack#831

[wilwade]

Notes and Resources for working on this:

• https://github.com/httptoolkit/brotli-wasm
  • https://github.com/httptoolkit/brotli-wasm/blob/main/build.js
  • https://rustwasm.github.io/docs/wasm-pack/commands/build.html
• https://developer.mozilla.org/en-US/docs/Web/API/CompressionStream/CompressionStream
  • This gets us a better gzip without zlib
• https://esbuild.github.io/plugins/#webassembly-plugin
• This one is working: https://github.com/jungomi/xxhash-wasm/
• https://esbuild.github.io/content-types/#binary

WIP Branch: fix/compression-issues

[wilwade]

Should be fixed in https://github.com/LibertyDSNP/parquetjs/releases/tag/v1.8.2

Please reopen if it does not.