Allow custom babel config for content plugins

[vjpr]

🚀 Feature

Have you read the Contributing Guidelines on issues?

Yes

Motivation

I want to use tailwindcss in my docs/blog/pages.

You can see here the createMDXLoaderRule hardcodes the babel loader and doesn't allow configuring the babelOptions.

  function createMDXLoaderRule(): RuleSetRule {
        return {
          test: /(\.mdx?)$/,
          include: flatten(versionsMetadata.map(getDocsDirPaths))
            // Trailing slash is important, see https://github.com/facebook/docusaurus/pull/3970
            .map(addTrailingPathSeparator),
          use: compact([
            getCacheLoader(isServer),
            getBabelLoader(isServer),
            {

export function getBabelLoader(
  isServer: boolean,
  babelOptions?: TransformOptions | string,
): Loader {
  return {
    loader: require.resolve('babel-loader'),
    options: getBabelOptions({isServer, babelOptions}),
  };
}

export function getBabelOptions({
  isServer,
  babelOptions,
}: {
  isServer?: boolean;
  babelOptions?: TransformOptions | string;
} = {}): TransformOptions {
  if (typeof babelOptions === 'string') {
    return {
      babelrc: false,
      configFile: babelOptions,
      caller: {name: isServer ? 'server' : 'client'},
    };
  } else {
    return Object.assign(
      babelOptions ?? {presets: [require.resolve('../babel/preset')]},
      {
        babelrc: false,
        configFile: false,
        caller: {name: isServer ? 'server' : 'client'},
      },
    );
  }
}

Pitch

The easy fix is to add babelOptions to the plugin config and pass it through.

---

Issue was identified thanks to: ben-rogerson/twin.macro#125 (comment)

[vjpr]

Workaround

This webpack plugin will modify all mdx loaders to use a custom babel config.

module.exports = function customWebpackPlugin(context, options) {
  return {
    name: 'custom-webpack-plugin',
    configureWebpack(config, isServer, utils) {
      modifyBabelConfigForDocusaurusContentDocsPlugins(config, isServer, utils)
	}
  }
}

function modifyBabelConfigForDocusaurusContentDocsPlugins(config, isServer, utils) {
  const mdxRegex = /(\.mdx?)$/
  const matchingRules = config.module.rules.filter(
    rule => rule.test.toString() === mdxRegex.toString(),
  )

  //const {getBabelLoader} = utils

  matchingRules.map(rule => {
    const babelLoaders = rule.use.filter(use => use.loader.match('babel-loader'))
    babelLoaders.map(loader => {
      loader.options.configFile = join(process.cwd(), 'babel.config.js')
      delete loader.options.presets
      delete loader.options.babelrc
    })
  })
}

[slorber]

Hi,

We already support the babel.config.js file out of the box:
https://v2.docusaurus.io/docs/next/configuration/#customizing-babel-configuration
Have you tried it?

You are right that it seems it's not used by the mdx loaders of the content plugins, leading to your custom config not being applied when loading mdx files.

Your custom babel config would only work when loading .js/jsx components, not .md/.mdx files.
Can you confirm it's the case?

If we add support for the babel config to the mdx loaders, is it good enough for you? Or do you see usecases where the babel config should be different per plugin, or between .js/mdx?

---

As your usecase seems to be using Tailwind with Twin macros (#3994 (comment)), I must say I'm not sure you would be able to use the tw or css prop working in MDX, as these features rely on using a JSX pragma. You can only use one JSX pragma at the same time, and MDX already adds its pragma, so it's likely to conflict with other css-in-js libs that relies on pragma to add non-std props (emotion/styled-components/twin/theme-ui... ).
Let me know if it works, I'd be interested to understand how 😅

[vjpr]

you would be able to use the tw or css prop working in MDX,

It does seem to work for me, doing something like:

import 'twin.macro'

<div tw="bg-gray-50 pt-12 sm:pt-16">
  <div tw="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
  </div>
</div>

Your custom babel config would only work when loading .js/jsx components, not .md/.mdx files.

Yep. I mistakenly thought .mdx could handle JS code, but it only supports import and JSX elements.

So the custom babel config is useless for much else...but still it does seem to work for mdx.

[slorber]

Sorry but I'm a bit confused 😅
what does work and what does not? What do you expect me to do?

We have a brand new http://new.docusaurus.io/ to easily create repro cases. If you can show me exactly what I should add support for with a failing sandbox, that would help me implement the missing bits needed

[vjpr]

Ideally I would like to be able to pass in a custom createMDXLoaderRule function, and therefore have every function used by this function to be exported...so I can swizzle it.

Will look at a repro soon.

[vjpr]

One of the most pressing configs I need actually in a monorepo context is the ability to exclude custom paths from each plugin-content-docs loader.

Here is a dump of my webpack config for my root docs plugin (generated via https://www.npmjs.com/package/webpack-config-dump-plugin). I need to be able to modify the exclude so that it ignores paths where other plugin-content-docs plugins can be found to avoid mdx files being processed twice.

For an example of my monorepo structure see here (#4055 (comment)).

      {
        test: /(\.mdx?)$/,
        include: [ '/xxx' ],
        use: [
          {
            loader: '/xxx/node_modules/.pnpm/[email protected][email protected]/node_modules/cache-loader/dist/cjs.js',
            options: { cacheIdentifier: 'cache-loader:4.1.0false' }
          },
          {
            loader: '/xxx/node_modules/.pnpm/[email protected]_80cd7b8149d93293e480a9a7c4e34482/node_modules/babel-loader/lib/index.js',
            options: {
              configFile: '/xxx/repos/live/packages/apps/docs/babel.config.js',
              caller: { name: 'client' }
            }
          },
          {
            loader: '/xxx/node_modules/.pnpm/@docusaurus/[email protected][email protected][email protected]/node_modules/@docusaurus/mdx-loader/src/index.js',
            options: {
              staticDir: '/xxx/repos/live/packages/apps/docs/static'
            }
          },
          {
            loader: '/xxx/node_modules/.pnpm/@docusaurus/[email protected][email protected][email protected]/node_modules/@docusaurus/plugin-content-docs/lib/markdown/index.js',
            options: {
              siteDir: '/xxx/repos/live/packages/apps/docs',
              sourceToPermalink: {
                '@site/../../../../../docs/_readme.md': '/xxx/docs/_readme',
              },
              versionsMetadata: [
                {
                  versionName: 'current',
                  versionLabel: 'Next',
                  versionPath: '/thirtyfive',
                  isLast: true,
                  routePriority: -1,
                  sidebarFilePath: '/xxx/docs/sidebar.js',
                  docsDirPath: '/xxx'
                }
              ]
            }
          }
        ],
        exclude: [ /node_modules/, /\/repos/, /\/.dev/, /\/tmp/ ] <----- I had to manually modify the webpack config to add these excludes.
      },

This doesn't seem to be set in the plugin-content-docs. It must be set in @docusaurus/core perhaps?

      function createMDXLoaderRule(): RuleSetRule {
        return {
          test: /(\.mdx?)$/,
          include: flatten(versionsMetadata.map(getDocsDirPaths)),
          use: compact([
            getCacheLoader(isServer),
            getBabelLoader(isServer),
            {
              loader: require.resolve('@docusaurus/mdx-loader'),
              options: {
                remarkPlugins,
                rehypePlugins,
                beforeDefaultRehypePlugins,
                beforeDefaultRemarkPlugins,
                staticDir: path.join(siteDir, STATIC_DIR_NAME),
                metadataPath: (mdxPath: string) => {
                  // Note that metadataPath must be the same/in-sync as
                  // the path from createData for each MDX.
                  const aliasedPath = aliasedSitePath(mdxPath, siteDir);
                  return path.join(dataDir, `${docuHash(aliasedPath)}.json`);
                },
              },
            },
            {
              loader: path.resolve(__dirname, './markdown/index.js'),
              options: docsMarkdownOptions,
            },
          ]),
        };
      }

      // Suppress warnings about non-existing of versions file.
      const stats = {
        warningsFilter: [VERSIONS_JSON_FILE],
      };

      return {
        stats,
        devServer: {
          stats,
        },
        resolve: {
          alias: {
            '~docs': pluginDataDirRoot,
          },
        },
        module: {
          rules: [createMDXLoaderRule()],
        },
      };
    },
  };
}