[JS] Unable to build with webpack with strictExportPresence: true

[tarekrached]

This took me a while to track down, but the long and short of it is that the apache-arrow npm module can't be used in webpack with the strictExportPresence webpack config flag set to true.

This manifested when I tried using apache-arrow in a create-react-app application - it worked fine in the development mode, but failed when I tried to build the production bundle. (Note that this was for [email protected] - in the nearly-completed react-scripts@2, dev mode fails, too.)

I have a minimal reproduction of this issue here.

The following may be related:

• ES2015 module issue with transpileOnly + decorators + interfaces TypeStrong/ts-loader#653
• warning on emitDecoratorMetadata for interface in esnext microsoft/TypeScript#18008

[tarekrached]

For context, I ran into this when I was trying to use uwdata/falcon in a create-react-app application.

As it currently stands, this issue precludes the use of the apache-arrow npm package, or any package that depends on apache-arrow, in any create-react-app application.

[wesm]

@trxcllnt @TheNeuralBit seems like we should try to get a new JS release out there. Could the data refactor wait until 0.5.0?

[trxcllnt]

@wesm yes definitely, I think that was the plan all along. I've also run into this issue with webpack, so we should fix this before 0.4.0. I'm not entirely sure how this webpack flag works, since the TS does compile down to compliant syntax for CommonJS and ESModules:

Arrow.js:

const table_1 = require("./table");
exports.Table = table_1.Table;

and the corresponding one in the ESModules file Arrow.mjs:

import { Table, CountByResult } from './table';
/* ... */
export { Table, CountByResult };

@tarekrached do you have any ideas on what the fix would be?

[trxcllnt]

For the record my workaround has been to add apache-arrow to the list of modules that babel transpiles, but I recognize that's only feasible because I had to eject from CRA for other reasons as well. Doing that allows babel-plugin-transform-runtime to replace tslib generators with regenerator code, so we reduce the bundle size a little bit:

    webpackConfig.module.rules[0].oneOf.push({
        test: /\.(js|jsx|mjs)$/,
        loader: 'happypack/loader?id=babel',
        exclude: webpackConfig.module.noParse,
        include: [
            , paths.appSrc
            , /node_modules\/ix/
            , /graphistry\/apps/
            , /node_modules\/apache-arrow/
            , /node_modules\/symbol-observable/
        ].filter(Boolean),
    });

[tarekrached]

@trxcllnt I don't have a clue what the fix would be, sorry. I tried messing around with some of the ts compiler settings in js/tsconfig and rebuilding, but to no avail. The TypeScript build process is something of a mystery to me.

FWIW it looks like this no longer manifests in the hot-off-the-presses [email protected]. npm start and npm run build both work fine, and they both failed in [email protected]. I'm not clear on what changed, as strictExportPresence is still set to true, so I'm not confident that it won't pop up again in future releases.

[tarekrached]

I think the relevant react-scripts change may be here, in which they treat *.mjs files specially.

facebook/create-react-app@3ae3cf3#diff-d2bddbd3bfb7051f9381474c844674fa

I say this because the error that surfaced in npm start was:

./node_modules/@apache-arrow/es2015-esm/type.js 27:18-29
  'flatbuffers' does not contain an export named 'flatbuffers'.

and the flatbuffers source is node_modules/flatbuffers/js/flatbuffers.mjs

[mlyknown]

everybody has some solution?

[tarekrached]

I spoke too soon - @apache-arrow/es2015-esm compiles, but flatbuffers is undefined:

This is with react-scripts 2.0.1 and 2.0.2

[tarekrached]

I think this issue in apollo-upload-client is related. AFAICT the issue stems from a named export in an upstream non-esm module.

also possibly related: webpack/webpack#7973 Broken cjs & esm interop

[wesm]

Any next steps here after the JS internals refactor, or can this be closed?

[trxcllnt]

I believe the original issue in this thread has been resolved by create-react-app updates. There's still a separate Webpack 4 bug that causes webpack to incorrectly compile when a project has CommonJS and ESM side-by-side. Apparently it has been fixed in Webpack 5, but create-react-app is still on Webpack 4 until Webpack 5 is out of alpha.

There is a workaround for Webpack 4 that I've tested works, so I've filed a report with CRA about rolling back to an earlier config that worked in 4. In the meantime, I've been using the @apache-arrow/esnext-cjs build in Graphistry. It may be worth switching the main apache-arrow module to be one of the CommonJS builds for this next release, then following up when these external issues are resolved.

[wesm]

OK, if there is an action item or something to follow up on please open a JIRA issue. Closing this for now