feat(compartment-mapper): CommonJS with lexical analysis

[kriskowal]

This change reintroduces support for CommonJS modules, but using a lexical analyzer instead of a full-parser analyzer. This reuses the analyzer built by @guybedford for Node.js, but including the import graph analysis by @developit in nodejs/cjs-module-lexer#10.

Fixes #501
Refs #655

The change consists of three parts.

1. a breaking change to SES changes the API for non-ESM static module records such that they are required to statically provide a list of their exports. This in turn enables third-party modules to participate fully in named imports and for their exports to be reexported by intermediate ESMs,
2. the creation of a fork of the CommonJS module lexer,
3. the integration of the lexer in the Compartment Mapper. This in turn consists of reverting the prior change that disabled CommonJS support because the parser was too bulky of a dependency, and then replacing and updating the lexical analysis. Notably, the tests do not change.

Individual commits are reviewable. If requested, I can arrange for this change to be distributed into three or more smaller reviews. This is currently proposed as draft so review can begin while I increase the test coverage over over the lexer.

[kriskowal]

Tests pass and working end to end. Will follow-up with more commits to increase test coverage and catch up with changes to the original lexer that have landed since the PR I forked from.

[erights]

If requested, I can arrange for this change to be distributed into three or more smaller reviews.

@kriskowal , great to see this! But yes, please do break it up. Thanks.

[kriskowal]

This change is now reduced to the last of three parts, a small adjustment to the compartment mapper that finishes the job. I choose you @michaelfig.

[michaelfig]

LGTM! Just out of curiosity, is the case where MJS uses the CJS affordances handled?

Example, trivially:

import something from './other';
console.log(__dirname, something);
export const foo = require('soso');

That's a feature we use in some of our -resm packages, though not as trivially.

[kriskowal]

LGTM! Just out of curiosity, is the case where MJS uses the CJS affordances handled?

Example, trivially:

import something from './other';
console.log(__dirname, something);
export const foo = require('soso');

That's a feature we use in some of our -resm packages, though not as trivially.

The MJS module type does not support that particular misfeature since it’s not compatible with Node.js (creates an expectation that it ought to work though it won’t). This is very tricky. On the one hand, we could introduce a resm module type that differs only in that this misfeature is supported, but while it does create a bridge from RESM to Endo, it leaves no bridge to Node.js. I’ll mull on this.