Consider generating multiple bundles in favor of modern browsers

[kripod]

According to this article, modern browsers may profit from bundles loaded with support for the ES2015 module syntax. Using the methods described in the article may also reduce server load because of the reduced bundle sizes.

[KyleAMathews]

Huh super interesting! Would love to see someone set this up on gatsby! Should be doable as a plugin.

Btw, the main benefit appears to be that you don't have to polyfill for newer browsers so the code loaded is significantly smaller.

[jquense]

That article seems confused (or I am)...the main jist is to not compile modules via babel. The problem with that tho is that you don't get a bundle out that has uncompiled es modules, you get a compiled webpack bundle with no modules. Babel won't compile the modules but webpack will. So you can use the "modern" babel config with any browser, in fact the v2 gatsby branch (which uses webpack2) already does this.

[jquense]

ahhhh I get it sorry i'm the dense one. The point is just that you can simplify feature detection by checking script module tags?

[KyleAMathews]

@davidtheclark over at Batfish is working on a PR for a similar feature mapbox/batfish#188

This technique seems really promising.

We'd need to see how much time it adds to the build but if it's not that much more, I'd definitely consider turning it on by default. If the extra build time hurts enough, we could make it optional.

[robwierzbowski]

@jquense I'm really into this approach, and have been applying parts of it in prod code. To summarize my understanding:

• Transpiled code is often larger and less performant at runtime than es6+ code.
• script type="module" is a good "cut the mustard" line for a lot of es6+ features. Remember cut the mustard from maybe 5-6 years ago?
• We should still bundle our modules
• We can create a bundle with much less transpiling for modern browsers, and use script type="module" as a sneaky feature detect to load a minimally transpiled bundle on newer browsers, and a more transpiled / es5 bundle on older browsers.

[KyleAMathews]

Yup! And per our parallel polyfill discussion, the script type="module" would need pretty minimal polyfills and be very cheap to deliver them over CDN.

[davidtheclark]

I ended up holding off on that Batfish PR but implemented the plan on another build system tool — with the idea of bringing it into Batfish after Babel 7 is released. It worked well overall, but there were two hiccups:

• Minification becomes more complicated, because you want to minify your ES2015+ bundle without having to compile it to ES5. Uglify has been moving pretty fast to catch up to ES2015+, though, so this might not be much of an issue anymore. Seems to be working ok for me.
• You have to run the Webpack build twice, which is kind of a bummer.

[robwierzbowski]

Yeah, the build twice is a pain. Webpack persistant caching might hep that, depending on how it’s implemented.

Babel-minify worked well for me too. Not quite as small as uglify es branch, but not as many edge cases either.

[mquandalle]

For reference Meteor 1.7 introduced the ability to generate two different JavaScript bundles, one for legacy browsers that includes JS polyfills for features that aren't natively implemented by the browser, and one for modern browsers without these polyfills. The rationale for the Meteor design is exposed in meteor/meteor#9439 and https://blog.meteor.com/meteor-1-7-and-the-evergreen-dream-a8c1270b0901.

[KyleAMathews]

Cool! Super nice to see Meteor is doing this. This is a very high priority to ship with v2.

[benjamn]

@KyleAMathews I'd be happy to discuss how this might work in Gatsby! It's a nice selling point for Meteor right now, but I think it's safe to say Gatsby's users have different needs, and are not about to switch to Meteor.

Most of my thoughts are in the blog post that @mquandalle linked above, but here's a slightly shorter explanation that I recently typed in a FB comment:

Here's the gist: In previous versions of Meteor, we had only one set of static assets (JS, CSS, etc.), and those assets needed to work in any browser that we claimed to support.

In Meteor 1.7, these assets have become "legacy," which means they still work perfectly well in any browser that we support. Of course, in order to support older browsers, the JS bundle needed to be compiled using Babel, and the resulting code was larger, messier, less native.

In Meteor 1.7, we wanted to start shipping a smaller, less messy, more native bundle tailored for "modern" browsers that have native support for a growing set of ECMAScript features. To do that, we considered each polyfill and Babel plugin that we were previously using, and asked what would happen if we just stopped using that polyfill or plugin. Of course, removing a plugin that handles class syntax means your bundle can only run in browsers that natively support class syntax, so there's a cost to removing each plugin. Removing polyfills and plugins narrows down the set of browsers that can be considered "modern" (which just means it's safe to send them whatever modern bundle we built).

So that's essentially where the modern bundle comes from: it's a copy of the legacy bundle, with as many polyfills and plugins removed as we felt comfortable removing. For every feature that we stopped simulating in older browsers, we found the minimum version of each browser that natively supports that feature, so only those browser versions are eligible to receive the modern bundle. More precisely, to receive the modern bundle, a browser needs to meet the minimum version for all the features that we no longer compile.

You might think that removing, say, half of the plugins would result in a bundle that only a very small percentage of browsers could handle, but the amazing thing is that more than 80% of the world's browsers are "evergreen," and they all support basically the whole ECMAScript standard, so you have to remove a whole lot of plugins before you end up with a bundle that they can't handle. We'd be in a very different (worse) situation if every major browser supported some arbitrary partial set of ECMAScript features. Luckily, the evergreen browsers are in vigorous agreement.

Right now, the most bleeding-edge feature that we leave uncompiled in the modern bundle is probably async/await. Yes, that was just finalized in 2017, but all the evergreen browsers already support it. At some point we'll be able to stop compiling things like decorators and public/private class properties, because enough browsers will support those features natively, but we're not there yet.

Sniffing user agents can be dicey, but if the server is ever not 100% confident about the browser/version of a client, we can always fall back to the legacy bundle.

[benjamn]

One quick question: does Gatsby have any mechanism for delivering different HTML to different browsers, or is the <script type="module">/<script nomodule> (for differentiating between JS bundles) the only trick a static site can use?

[m-allanson]

Gatsby probably needs to rely on <script type="module">/<script nomodule>

[benjamn]

I think that's a perfectly reasonable strategy, though I just finished tweeting some thoughts on the limits of that approach: https://twitter.com/benjamn/status/1005104014944538624

[KyleAMathews]

Vue is shipping with this as an optional mode for builds https://cli.vuejs.org/guide/browser-compatibility.html#modern-mode

[KyleAMathews]

This is how vue-cli does modern builds https://github.com/vuejs/vue-cli/blob/38efc032e5c0d5ab8dd74602c1fb100fef62d766/packages/%40vue/cli-service/lib/webpack/ModernModePlugin.js

[davilima6]

This is the article that pulled me into this topic: https://nystudio107.com/blog/an-annotated-webpack-4-config-for-frontend-web-development It doesn't use HtmlWebpackPlugin (at least not for that) but is still a great example of a complete use of multi-target compilations: https://github.com/nystudio107/annotated-webpack-4-config/

[m-allanson]

@KyleAMathews and @DSchau you were working on this recently I think? Could you post an update of where you got to - was there some issues with handling multiple webpack configs?

[DSchau]

@m-allanson thanks for tagging us in!

We did. Here's where we ended up (with more work on this probably coming this week):

The current state of the work is here.

What we'll need to do is do two webpack builds (ideally in parallel, utilizing something like jest-worker), one with a modern config, and one with the legacy config.

Our approach is tentatively going to be opt-out, e.g. gatsby build --legacy, because we can't really see (currently!) why a dev wouldn't want this behavior.

Current blockers/issues/things to solve:

• Parallelizing build; we'll probably want to do this for performance reasons
  • See Kyle's tweet for some more info here
  • We can build both in the same process, but it's possible this may introduce significantly slower builds for people, so we'll want to weigh the cost of this
• Update the static-entry.js file in cache-dir to support module nomodule script tags
  • Additionally check out the Vue Safari fix and implementation
• Testing, testing, testing
  • We'll want to profile this, validate it works as expected, minimal build performance regression (or none at all), etc. etc.

Additional things that should probably be solved:

• Transpiling our dependencies (e.g. gatsby-image, gatsby-link)
  • We currently publish these as CommonJS - to get some more savings, we should publish a modern build of this (using microbundle?) and then these will be minimally transpiled with the modern build, and then certain features will be transpiled for the legacy build
  • More broadly speaking, could be nice to start transpiling node_modules as a whole, but that's a separate conversation

I think this is a decent intro on the current state of things. More to come soon!

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[ronva]

Hey, sorry about the radio silence. A bunch of stuff came up resulting in me not being to continue working on this feature. Still really hoping to see this get done, please let me know if there's anything I can help with!

[DSchau]

See: #12761 for a WIP PR @wardpeet and I worked on to use microbundle to publish our browser dependencies (e.g. gatsby-image, gatsby-link) as es and cjs bundles.

We're waiting for some upstream changes to microbundle, then we can merge those in!

This will get us some benefits as far as file size shipped to the browser to transpile less code. More to come!

[wardpeet]

slowly making microbundle ready for modern builds
developit/microbundle#362

[codepunkt]

Is there a status update on this?

[KyleAMathews]

more links

• [RFC] - Module/nomodule for NextJs vercel/next.js#7563 (comment)
• https://github.com/babel/preset-modules/blob/master/README.md

[jquense]

Has anyone identified even the correct pattern for doing this in webpack? I've wanted to set this up multiple times but just completely unsure waht the Right Way to even integrate it into webpack is

[developit]

@jquense If you're already running a MultiCompiler, the best solution is to add an additional top-level compiler pass. Otherwise, babel-esm-plugin performs the module build using a Child Compiler, which is runs in parallel to the parent compilation to a limited extent.

The problem with all current approaches is that they don't extend the benefits of modern JS to code obtained from node_modules. Thankfully, there has recently been some solid movement towards a standardized mechanism for exposing modern JS in npm packages, and I'm hoping to have a demonstration of this soon. It will take a while for packages to adopt the approach, but shipping it as a default in microbundle and tsdx can serve as a nice head-start.

[abdullahe]

I ended up here while searching how to get rid of 80kb of polyfill.js from the bundle. :) I guess there is not an easy solution for newbies yet.

[abdullahe]

Hi @ascorbic . Thanks for checking and letting me know. I didn't know browser selectively loads chunks from a bundle. I thought it loads whatever is included in the bundle, so I was trying to minimize it.

Sorry for taking your time. 🙏

[ghost]

I have this issue as well and when running lighthouse I get the message that "Avoid serving legacy JavaScript to modern browsers, Polyfills and transforms enable legacy browsers to use new JavaScript features. However, many aren't necessary for modern browsers." Is it possible to avoid using polyfill at all? (#28736)

[arielperez82]

I'm seeing the same as @devquestionz in my lighthouse audit and was hoping to find a way to exclude them.

[robmarshall]

Also seeing this

[ascorbic]

Please do check whether the polyfill is actually being loaded in modern browsers, rather than just showing in the bundle analyzer. The bundle analyzer shows all bundles, including those that aren't loaded by modern browsers. If anybody is able to provide a reproduction where a modern browser is loading the polyfills then please provide a link.

[AbdallahAbis]

any updates regarding this, I'm using "gatsby": "^2.32.8" and I'm getting this on Google Page Speed.

is there a way to Differential Serving in Gatsby?

cc @LekoArts @wardpeet @KyleAMathews

[kb1995]

Any updates on this? I'm using "gatsby": "4.19.2", but this is still a problem