Module with Top-level await transformed incorrectly during server build

[fawazahmed0]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Prerequisites

• I'm using the latest version of Docusaurus.
• I have tried the npm run clear or yarn clear command.
• I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• I have tried creating a repro with https://new.docusaurus.io.
• I have read the console error message carefully (if applicable).

Description

Hi, I have enabled top-level await support in webpack config, and things work fine during docusaurus run, but fails during docusaurus build

This is my mdx content:

---
title: Markdown page example
---
export const test = await Promise.resolve('hello') 


<div>{test}</div>

Reproducible demo

https://stackblitz.com/edit/github-7mt6ad?file=src/pages/test.mdx

Steps to reproduce

1. Enable top-level await support in webpack by setting experiments.topLevelAwait to true using configureWebpack (see plugins at docusaurus.config.js )
2. Try using top-level await during docusaurus run, things work fine.
3. It fails during docusaurus build

Expected behavior

top-level await should also work during build process

Actual behavior

top-level await doesn't work during build process

[INFO] [en] Creating an optimized production build...

✔ Client
  

✖ Server
  Compiled with some errors in 18.35s

[ERROR] Docusaurus server-side rendering could not render static page with path /test.


TypeError: Cannot read properties of undefined (reading 'title')
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.

Your environment

• Docusaurus version used: 2.0.0-beta.18
• Operating system and version (e.g. Ubuntu 20.04.2 LTS): windows 10

Self-service

• I'd be willing to fix this bug myself.

[Josh-Cena]

Related to #6520

We can investigate this—my hypothesis is that it has something to do with the static site generation plugin, which probably doesn't wait for the top-level promise to resolve. No guarantee that it will be fixed, though, since it's not our own code.

[Josh-Cena]

For some reason, TLA results in the default export being a promise, instead of the entire module being a promise. In order to investigate, I logged the output of what got loaded (the first prop passed to Loadable.Map). In CSR (both dev and prod):

But in SSR:

{
  __comp: {
    default: Promise {
      [Object [Module]],
      [Symbol(webpack exports)]: [Object [Module]],
      [Symbol(webpack then)]: [Function (anonymous)]
    }
  },
  '__context.plugin': {
    name: 'docusaurus-plugin-content-pages',
    id: 'default',
    default: { name: 'docusaurus-plugin-content-pages', id: 'default' }
  },
  config: Object [Module] { default: [Getter] }
}
Warning: React.createElement: type is invalid -- expected a string (for built-in components) or a class/function (for composite components) but got: object. You likely forgot to export your component from the file it's defined in, or you might have mixed up default and named imports.

I will spin up an isolated Webpack config to see who that is to blame (Babel? Webpack? React-loadable?)

[Josh-Cena]

I strongly suspect this has something to do with babel-plugin-dynamic-import-node. Maybe it converts import() to something like { default: Promise.resolve(require(<path>)) }? I need to check their implementation. I know 2.3.3 has broken some stuff for us.

Edit. Seems that's indeed the case: playground The reason why it puts it inside a default is because require returns a promise, which Babel doesn't know to await, so it can't see the __esModule marker...
So the next question is, do we actually need this, now that all Node versions we support also support dynamic import...

[Josh-Cena]

While trying to remove babel-plugin-dynamic-import-node I'm running into more problems. Here's my change:

$ git diff main packages/docusaurus/src/babel/preset.ts
diff --git a/packages/docusaurus/src/babel/preset.ts b/packages/docusaurus/src/babel/preset.ts
index 3dd351ef9a..1fc764d7da 100644
--- a/packages/docusaurus/src/babel/preset.ts
+++ b/packages/docusaurus/src/babel/preset.ts
@@ -62,10 +62,6 @@ function getTransformOptions(isServer: boolean): TransformOptions {
           absoluteRuntime: absoluteRuntimePath,
         },
       ],
-      // Adds syntax support for import()
-      isServer
-        ? require.resolve('babel-plugin-dynamic-import-node')
-        : require.resolve('@babel/plugin-syntax-dynamic-import'),
     ],
   };
 }

And I attempted to build a site with nothing but src/pages/index.js containing the following:

export const text = await Promise.resolve("Hello");

export default function Home() {
  return text;
}

Surprisingly, the landing page compiled successfully (edit: it didn't; it silently failed and outputs an HTML file containing nothing but an error banner), but the 404 page gave the error:

[ERROR] Docusaurus server-side rendering could not render static page with path "/404.html".
Error: Cannot find module './assets/js/763.3e8c76c2.js'
Require stack:
- main
    at Array.reduce (<anonymous>)
    at Array.map (<anonymous>)
[ERROR] Unable to build website for locale en.

The actual chunk for the 404 page is 763.d3922ca1.js.

I've seen the same errors in the past when I'm setting up SWC, but I don't have the patience to look into it since it looks super obscure. My hypothesis is that it's because the same file is compiled differently, leading to different hashes (? I decided to not touch it at this point. If anyone wants to look into it, help appreciated.

[Josh-Cena]

I actually dug into the server bundle output (before SSG plugin kicks in), and the "module not found" is thrown from here:

/* webpack/runtime/get javascript chunk filename */
(() => {
	// This function allow to reference async chunks
	__webpack_require__.u = (chunkId) => {
		// return url for filenames based on template
		return "assets/js/" + ({"115":"4825d8ed","237":"1df93b7f"}[chunkId] || chunkId) + "." + {"115":"17431192","237":"9726a329","763":"3e8c76c2"}[chunkId] + ".js";
	};
})();

Note that 763 is mapped to 3e8c76c2, which differs from its actual hash that's output to disk (from the client build), because the Babel output is different, causing the content hash to be different. When we had the babel-plugin-dynamic-import-node, nothing similar to that has been outputted, because Webpack didn't do code-splitting with require, so there were no "chunks" at that time. I guess we just have to (a) make the server build output code-splitted files and later clean them up or (b) stop server build from doing code-splitting and referencing physical assets even with dynamic import (?)

Edit. The problem is that the SSG webpack plugin kicks in too early: before the dynamically imported code gets emitted. Here's a minimum repro:

// entry.js
export default function render() {
  return import("./comp.js").then(({ val }) => val);
}

// comp.js
export const val = "Hello";

// webpack.config.js
const path = require("path");
const StaticSiteGeneratorPlugin = require("@slorber/static-site-generator-webpack-plugin");

module.exports = {
  target: "node18.0",
  mode: "development",
  name: "test",
  entry: {
    main: path.resolve(__dirname, "./entry.js"),
  },
  output: {
    clean: true,
    pathinfo: false,
    libraryTarget: "commonjs2",
    globalObject: "this",
    path: path.resolve("build"),
    filename: "server.bundle.js",
    chunkFilename: "assets/js/[name].[contenthash:8].js",
    publicPath: "/",
    hashFunction: "xxhash64",
    libraryTarget: "commonjs2",
  },
  plugins: [
    new StaticSiteGeneratorPlugin({
      entry: "main",
      paths: ["/"],
    }),
  ],
  optimization: {
    splitChunks: false,
  },
};

When you comment out the SSG plugin, everything gets emitted, including the dynamic chunks; however, the above code throws: Cannot find module './assets/js/comp_js.b71b1699.js'

Maybe we can fix it in the SSG plugin?