How to improve webpack splitChunks heuristics?

[slorber]

@Gatsby suggested me on Twitter to open an issue on this subject.

I think it would be nice to document how an user can enhance gatsby/webpack default code splitting heuristics to achieve better code splitting.

The Webpack splitChunks documentation:
https://webpack.js.org/plugins/split-chunks-plugin/

The current Gatsby config is:
https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/utils/webpack.config.js#L469

      splitChunks: {
        name: false,
        cacheGroups: {
          // Only create one CSS file to avoid
          // problems with code-split CSS loading in different orders
          // causing inconsistent/non-determanistic styling
          // See https://github.com/gatsbyjs/gatsby/issues/11072
          styles: {
            name: `styles`,
            // This should cover all our types of CSS.
            test: /\.(css|scss|sass|less|styl)$/,
            chunks: `all`,
            enforce: true,
          },
        },
      },

As we can see, apart from disabling css code splitting, Gatsby does not do much and relies on Webpack defaults.

I think it would be nice to document how a user can enhance those heuristics so that webpack can give more optimized outputs.

Also it's probably worth discussing what is an optimized webpack output in the first place, in the context of a Gatsby static site. We might favor multiple smaller bundles so that they can be better cached independently, or downloaded/parsed in parallel.

Webpack can't really know by himself how our code is going to change over time, and recipes or a plugin to tell him new heuristics could be nice.

I've been successful with this kind of config:

exports.onCreateWebpackConfig = ({ stage, actions, getConfig }) => {
  if (stage === `build-javascript`) {
    const config = getConfig();

    // Optimize the default code splitting config by putting heavy things
    // that changes independently on separate bundles
    config.optimization.splitChunks.cacheGroups = {
      ...config.optimization.splitChunks.cacheGroups,
      vendors: {
        test: /[\\/]node_modules[\\/]/,
        enforce: true,
        chunks: 'all',
        priority: 1,
      },
      translations: {
        test: /[\\/]translations[\\/]/,
        enforce: true,
        chunks: 'all',
        priority: 1,
      },
    };

    // Ensure Gatsby does not do any css code splitting
    config.optimization.splitChunks.cacheGroups.styles.priority = 10;

    actions.replaceWebpackConfig(config);
  }
};

I want to do that because I want separate bundles for my page, my libs and the translations. Generally, not all 3 are going to change together so if only translations are changing, I don't necessarily want users to have to redownload the other 2. If I do a lib upgrade, I don't necessarily want users to redownload the translations and may prefer them to hit the cache. With default heuristics I get a big common bundle that contains both translations and libs. I'd rather get them cached independently and downloaded/parsed in parallel.

I'm not an expert with all this, but I think it's worth discussing how we can get better performances by documenting how to fine tune the code splitting with usecases/recipes.

Also, maybe it would be nice to create an official plugin to easily do this tunning, and ensuring that we don't enable the css code splitting by mistake (which Gatsby disable on purpose).

Like:

{ 
  resolve: "gatsby-plugin-splitChunks",
  options: {
    cacheGroups: ["node_modules","src/something"],
  },
},

What do you think?

Related issues:
#10965 (comment)
#13906 (comment)
#13977
#15302

Also worth exploring what NextJS is doing and looking at related issues. There's a new "granularChunks" option that got merged recently
https://github.com/zeit/next.js/blob/5a8ed815fc605b323127d82a0952c312348e7340/packages/next/build/webpack-config.ts#L192

Also worth to consider if default webpack code splitting heuristics are good enough. The new granular option of Next seems to have an interesting setting that adapts according to the number of pages: vercel/next.js#7631

Maybe it's worth to do like Next and offer some "code splitting presets", like default / granular / other...

---

Edit some changes have already been made in this PR: #17093

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜