esm import cjs in third-party-package will get exports.default instead of module.exports

[lisonge]

npm init -y
npm i esbuild node-html-parser typescript
npx tsc --init
echo 'import { parse, HTMLElement } from "node-html-parser";console.log({ parse, HTMLElement });' > test.ts
npx esbuild test.ts --bundle --outfile=test.js
node test.js

will be output

{ parse:undefined, HTMLElement:undefined }

example https://github.com/lisonge/test-cli/runs/3958824691?check_suite_focus=true#step:10:1

probable cause taoqf/node-html-parser#166 (comment)

I can also delete exports in package.json to solve it, but that is not convenient,

Is there any other solution?

[evanw]

You have stumbled into an unfortunate interoperability mess. It happens when an ES module imports a CommonJS module with the default property present and the __esModule property set to true. Different tools just do different things for historical reasons, so basically you should just avoid ever doing this to maximize compatibility. The node-html-parser does this when node-html-parser/esm/index.js imports the default import from node-html-parser/dist/index.js, which causes this problem.

Background

Here's the historical context (at least from what I understand, although I could be wrong):

When ES modules first appeared, everyone used Babel to compile them to CommonJS so they could be evaluated. When converting ESM exports, all export names became properties on exports and then the special __esModule flag was set to true to indicate that the resulting CommonJS module used to be an ES module. When converting ESM imports, all import statements became a require() call and then if the special __esModule flag was set on the object returned by require(), the named ESM imports were matched up with properties on that object. So if you had both a default export and a default import in the original ESM code, everything would work fine.

Then node later came out with a way to import CommonJS from ESM, except they made it work differently. This caused all of these compatibility problems. The default import was always the value of the exports object no matter what. So when importing from a CommonJS module that was compiled from an ES module, you have to access default.default to get the original default export in node but just default to get the original default export in Babel. This mess could have been avoided if node had just followed the existing CommonJS-from-ESM semantics instead of doing something different, but I believe the problem was that the __esModule property is a run-time thing while ESM export names are a compile-time thing. Although ironically node is now statically analyzing CommonJS to generate ESM export names after all, so this actually would have been possible. Anyway that ship has sailed.

There are both a ton of old packages that were published to npm containing ES modules that expect to be evaluated the Babel way (since everyone used to have to compile with Babel before running ES modules), and also new packages being published to npm containing ES modules that expect to be evaluated the node way (since now you can just run ES modules in node without translation). Tools like esbuild, webpack, parcel, rollup, etc. need to decide if a given ES module is supposed to be run one way or the other way, and if they get it wrong either old packages break or new packages break. Each tool probably uses slightly different heuristics to try to guess which way to go. The heuristic currently used by esbuild is that the default export is the exports object (like node) except when exports.__esModule && 'default' in exports is true, in which case the default export is exports.default (like Babel).

Incompatibilities

Here is an example of the differences between tools. As you can see, esbuild currently matches rollup's behavior but otherwise every tool does something different:

	webpack	parcel	esbuild	rollup
import { parse } from "node-html-parser"	function parse()	function parse()	undefined	undefined
import x from "node-html-parser"	{ parse: function parse() }	function parse()	function parse()	function parse()

I can try to tweak esbuild's heuristics here although that will likely break someone else's package. A reasonable approach would probably be to try to replicate Webpack's behavior for this as much as possible since it's the most widely-used bundler. AFAIK they run in node-mode with the .mjs extension or with "type": "module" and otherwise run in Babel-mode. I'm not sure if they take into account "exports": { "import": {} } as is the case here or not. This would need more investigation.

How to fix this

There isn't a trivial way for you to deal with this on your own, since the presence of exports prevents you from directly importing the relevant files. You can bypass this with an esbuild plugin that intercepts the import of node-html-parser and resolves it to node-html-parser/dist/index.js. Here's an example esbuild plugin:

let fixNodeHtmlParserPlugin = {
  name: 'fix-node-html-parser',
  setup(build) {
    build.onResolve({ filter: /^node-html-parser$/ }, () => {
      return { path: require.resolve('node-html-parser') }
    })
  },
}

The author of the node-html-parser package could also fix this interoperability issue themselves by changing their code to avoid exporting default in the ESM source files that are then compiled to CommonJS files by TypeScript. But it sounds like they don't want to do this. So the interoperability issue with node-html-parser will likely persist.

There's a big discussion about this over at #532 for even more context.

[JulianKingman]

Dear @evanw, you are a gentleman and a scholar. You have saved me this day, and enlightened me on the source of my sorrows. I don't know how you got such a deep understanding of ESM modules and the various build behaviors, but I'm sincerely grateful that you chose to share it in so much detail here.