Rework variants to reduce bundle size #83

MattiasBuelens · 2021-07-21T16:39:39Z

This removes some variants and build artifacts to reduce the total download size of the npm package.

Remove all generated source maps.
Remove the ES2018 variant, which is now combined into the ES6 variant. This means that ReadableStream.prototype[Symbol.asyncIterator] won't correctly inherit from %AsyncIteratorPrototype%, but nobody cares about that. 😛
Remove unminified polyfill variants. All build artifacts for the polyfill variants are now always minified, while the ponyfill variants are always unminified.

Fixes #82.

To do:

~~Would it make sense to have ponyfill.mjs import from ponyfill.js and re-export the CommonJS module as an ES module, as documented here?~~
Maybe this is a good time to tackle Support importing as ESM in Node #51?

MattiasBuelens · 2021-07-21T16:46:29Z

With the current changes, we're at 182.0 kB packed, 1.1 MB unpacked. Still quite heavy...

npm pack

npm notice
npm notice package: [email protected]
npm notice === Tarball Contents ===
npm notice 1.1kB   LICENSE
npm notice 56.0kB  dist/polyfill.es6.min.js
npm notice 60.2kB  dist/polyfill.min.js
npm notice 187.5kB dist/ponyfill.es6.js
npm notice 198.1kB dist/ponyfill.js
npm notice 212B    es6/package.json
npm notice 2.4kB   package.json
npm notice 169B    ponyfill/es6/package.json
npm notice 148B    ponyfill/package.json
npm notice 329B    dist/types/tsdoc-metadata.json
npm notice 9.5kB   CHANGELOG.md
npm notice 7.0kB   README.md
npm notice 55.3kB  dist/polyfill.es6.min.mjs
npm notice 59.9kB  dist/polyfill.min.mjs
npm notice 170.2kB dist/ponyfill.es6.mjs
npm notice 180.2kB dist/ponyfill.mjs
npm notice 32.1kB  dist/types/polyfill.d.ts
npm notice 32.0kB  dist/types/ts3.6/polyfill.d.ts
npm notice === Tarball Details ===
npm notice name:          web-streams-polyfill
npm notice version:       3.0.3
npm notice filename:      web-streams-polyfill-3.0.3.tgz
npm notice package size:  182.0 kB
npm notice unpacked size: 1.1 MB
npm notice shasum:        31267b84b04aba59c868428e3a1fd5a216e105a7
npm notice integrity:     sha512-BVuy0GUcG+UCO[...]qtE78dlHSIY6Q==
npm notice total files:   18

jimmywarting · 2021-07-21T17:55:21Z

fyi, renaming, moving or removing different files/variants can cause a breaking change and would need a major change as ppl can depend on choosing their own variant to do the job

MattiasBuelens · 2021-07-21T19:04:21Z

I know, that's why I added a "semver-major" label. 😁

MattiasBuelens · 2021-08-10T23:01:21Z

Down to 119 KB packaged, 638.8 KB unpacked.

I'm now using code splitting on the ESM bundles, so polyfill[.es6].mjs imports directly from ponyfill[.es6].mjs. Unfortunately, I can't do this for the UMD bundles.

npm pack

npm notice
npm notice package: [email protected]
npm notice === Tarball Contents ===
npm notice 1.1kB   LICENSE
npm notice 56.2kB  dist/polyfill.es6.js
npm notice 60.4kB  dist/polyfill.js
npm notice 55.3kB  dist/ponyfill.es6.js
npm notice 59.7kB  dist/ponyfill.js
npm notice 200B    es6/package.json
npm notice 2.2kB   package.json
npm notice 169B    ponyfill/es6/package.json
npm notice 148B    ponyfill/package.json
npm notice 329B    dist/types/tsdoc-metadata.json
npm notice 9.5kB   CHANGELOG.md
npm notice 7.0kB   README.md
npm notice 2.1kB   dist/polyfill.es6.mjs
npm notice 2.4kB   dist/polyfill.mjs
npm notice 170.0kB dist/ponyfill.es6.mjs
npm notice 180.1kB dist/ponyfill.mjs
npm notice 32.0kB  dist/types/polyfill.d.ts
npm notice === Tarball Details ===
npm notice name:          web-streams-polyfill
npm notice version:       3.1.0
npm notice filename:      web-streams-polyfill-3.1.0.tgz
npm notice package size:  119.0 kB
npm notice unpacked size: 638.8 kB
npm notice shasum:        c3cfb0dba11b916c0134f8cf765548f2d978f7db
npm notice integrity:     sha512-PDktomxray+Oh[...]ZOr+EkDKujVVg==
npm notice total files:   17

jimmywarting · 2021-08-11T19:44:49Z

could the polyfill load the ponyfill instead?

Meaning ponyfill would just be as tiny as loading the ponyfill and assigning it to the global namespace...

wojpawlik · 2021-08-12T09:32:01Z

No it can't. Polyfill would need to synchronously import the ponyfill.

It'd be neat if a single UMD build could polyfill when loaded in browser, and ponyfill when loaded as CJS.

Windows 11 won't ship IE. Are ES5 builds really needed?

This is looking pretty good already.

wojpawlik · 2021-08-12T09:55:53Z

Would it make sense to have ponyfill.mjs import from ponyfill.js

Probably not. It wouldn't work in browsers. And Node.js ^12.20.0 || >=14.13.1 can already detect named exports in CJS (haven't tested UMD): nodejs/node#35249 (comment).

jimmywarting · 2021-08-12T10:49:38Z

Windows 11 won't ship IE. Are ES5 builds really needed?

IE11 is pretty much dead already... down at 0.57% https://gs.statcounter.com

This is looking pretty good already.

Agree, but still think there is room for improvements, but don't know how. How about minifying all files?

wojpawlik · 2021-08-14T16:26:39Z

npm notice 200B    es6/package.json
npm notice 169B    ponyfill/es6/package.json
npm notice 148B    ponyfill/package.json

could be replaced with https://nodejs.org/api/packages.html#packages_subpath_exports.

And CHANGELOG.md could be .npmignored.

MattiasBuelens · 2021-08-15T14:02:12Z

Thanks for the suggestions! 🙂

Indeed, I cannot have the UMD polyfill load the UMD ponyfill like I did with ESM, because there's no general/reliable way to do that.

I don't know if people expect ESM bundles to be human readable? Anyway, I can turn on minification unconditionally.

The company I work for also uses this polyfill, and we still support IE11... 😬 I'll check if we can make the polyfill ES6-only and set up some transpilation on our node_modules.

I've got an experiment branch that uses package entry points instead. I want to first check if bundlers like Webpack and Rollup handle these correctly before I commit to it.

jimmywarting · 2021-08-15T14:45:39Z

I don't know if people expect ESM bundles to be human readable

Doesn't matter to me so much. At least when it comes to spec'ed apis
I mostly expect them to say the function toString is native code. And I know what the code is suppose to do/output. I don't go so much into depths on built-in codes. Polyfills should not be tampered with anyways.
Much less so when typing exist to help out

Other packages yes. Then it is nice with readable code.

Minified code loads/parse faster and and have the potential for code optimization that makes it run faster.

So I'm +1 for minification

jimmywarting · 2021-08-15T14:49:00Z

Just speculation. But could the typing reference (or extend) the Dom implementation so d.ts could be reduced?

MattiasBuelens · 2021-08-15T20:29:26Z

And CHANGELOG.md could be .npmignored.

Apparently that's not possible. From the npm docs:

The following paths and files are never ignored, so adding them to .npmignore is pointless:

package.json

README (and its variants)

CHANGELOG (and its variants)

LICENSE / LICENCE

So CHANGELOG.md is always included in the package.

Just speculation. But could the typing reference (or extend) the Dom implementation so d.ts could be reduced?

That requires a /// <reference lib="dom" /> which is likely to cause issues for Node users, see microsoft/TypeScript#33111... 😬 It looks like a similar remark was raised in a recent change to node-fetch? Not sure if/how it was tackled there.

Looks like they're trying to tackle this by extracting the type definitions shared between Node and DOM into a separate file: microsoft/TypeScript#43972. That would solve the problem.

jimmywarting · 2021-08-15T21:21:45Z

Working a bit with deno nowdays... guess full urls don't work with microsofts own typescript yet

/// <reference lib="https://github.com/MattiasBuelens/web-streams-polyfill/blob/36c08de41aa3a5699afab717b0e64af13dada56f/x.d.ts" />

the alternative is hosting types at @types

MattiasBuelens · 2021-08-16T22:07:58Z

Minification is now enabled across the board. We're now at 87.0 kB packed, 401.8 kB unpacked.

npm pack

npm notice
npm notice package: [email protected]
npm notice === Tarball Contents ===
npm notice 1.1kB  LICENSE
npm notice 56.2kB dist/polyfill.es6.js
npm notice 60.4kB dist/polyfill.js
npm notice 55.3kB dist/ponyfill.es6.js
npm notice 59.7kB dist/ponyfill.js
npm notice 2.9kB  package.json
npm notice 329B   dist/types/tsdoc-metadata.json
npm notice 9.5kB  CHANGELOG.md
npm notice 7.0kB  README.md
npm notice 1.6kB  dist/polyfill.es6.mjs
npm notice 1.6kB  dist/polyfill.mjs
npm notice 54.6kB dist/ponyfill.es6.mjs
npm notice 59.5kB dist/ponyfill.mjs
npm notice 32.0kB dist/types/polyfill.d.ts
npm notice === Tarball Details ===
npm notice name:          web-streams-polyfill
npm notice version:       3.1.0
npm notice filename:      web-streams-polyfill-3.1.0.tgz
npm notice package size:  87.0 kB
npm notice unpacked size: 401.8 kB
npm notice shasum:        b0dbafc956733f5b64e8763077c326adb8039191
npm notice integrity:     sha512-8zEDoB88N8ZXM[...]Fgu/9aDufuvrg==
npm notice total files:   14

Package entry points seem to work fine in the latest versions of Rollup and Webpack, so I made the switch. For my own sanity, I've added tests for both bundlers and for require-ing it directly with Node. 😁

package.json

wojpawlik · 2021-08-17T18:47:56Z

What do dist/ponyfill.es6.js and dist/ponyfill.js do when loaded in a browser anyway?

package.json

MattiasBuelens · 2021-08-18T20:47:32Z

What do dist/ponyfill.es6.js and dist/ponyfill.js do when loaded in a browser anyway?

They can register with an AMD module loader (like RequireJS through define()). If no such module loader is found, they register a single global variable called WebStreamsPolyfill (defined here), so you can access the exported classes with new WebStreamsPolyfill.ReadableStream().

Do I expect anyone to actually be using this? Not really... 😛

MattiasBuelens · 2021-08-18T22:36:18Z

I made the ES6 ponyfill the default variant. If we're going to make breaking changes, we're going to really break things. 😛

Still not sure about the name of the polyfill variant though: import "web-streams-polyfill/polyfill" sounds... repetitive. Suggestions welcome!

There's not much of a difference in the package size: 86.2 kB packed, 397.0 kB unpacked.

npm pack

npm notice package: [email protected]
npm notice === Tarball Contents ===
npm notice 1.1kB  LICENSE
npm notice 59.7kB dist/polyfill.es5.js
npm notice 55.2kB dist/polyfill.js
npm notice 59.7kB dist/ponyfill.es5.js
npm notice 55.3kB dist/ponyfill.js
npm notice 2.8kB  package.json
npm notice 329B   dist/types/tsdoc-metadata.json
npm notice 9.5kB  CHANGELOG.md
npm notice 7.2kB  README.md
npm notice 59.5kB dist/ponyfill.es5.mjs
npm notice 54.6kB dist/ponyfill.mjs
npm notice 32.0kB dist/types/ponyfill.d.ts
npm notice === Tarball Details ===
npm notice name:          web-streams-polyfill
npm notice version:       3.1.0
npm notice filename:      web-streams-polyfill-3.1.0.tgz
npm notice package size:  86.2 kB
npm notice unpacked size: 397.0 kB
npm notice shasum:        debf2c6af0fb5115af08b9caa8ecdc8f4acee598
npm notice integrity:     sha512-tkJkzQcEgeVXW[...]NU+H64Wv0ibsg==
npm notice total files:   12

package.json

MattiasBuelens · 2021-08-19T22:06:39Z

Hmm, now that the polyfill variant does not export anything, I suppose it needs a different .d.ts to reflect that... It also looks like TypeScript might not yet support package entry points, see microsoft/TypeScript#33079. 😕

MattiasBuelens · 2021-08-27T21:41:51Z

Found a workaround for the type definitions using typesVersions. Now, import "web-streams-polyfill/polyfill" will have no exports, to match the run-time behavior. 😁

Even better: it will now augment TypeScript's built-in ReadableStream type with the new methods added by the polyfill. Right now, that's getReader({ mode: 'byob' }) for BYOB readers and values() for async iteration.

I would have liked to make new ReadableStream({ type: 'bytes' }) work too, but I don't think that's possible with how lib.dom.d.ts defines the global ReadableStream variable. 😞

Still not sure about the name of that entry point though. I've come up with a few alternatives:

import "web-streams-polyfill/install", to indicate that this will install the polyfill in the global scope
import "web-streams-polyfill/register", inspired by @babel/register and ts-node/register

"browser" is unnecessary if it's equal to "main".

Blocked by rollup/plugins#983

MattiasBuelens · 2021-09-06T20:21:50Z

Released version 4.0.0-beta.1, see release notes. Looking forward to your feedback! 😀

jimmywarting · 2021-09-06T20:24:00Z

Cool! Want to test it now, but guess i will do it tomorrow instead (22:23) here now.
+ i'm busy with another project i'm working on right now

jimmywarting · 2021-09-06T20:27:09Z

402 kB, That is a good reduction :)
(compared to prev 7.56 MB)

MattiasBuelens added the semver-major label Jul 21, 2021

MattiasBuelens self-assigned this Jul 21, 2021

MattiasBuelens changed the base branch from master to next July 21, 2021 21:59

MattiasBuelens force-pushed the reduce-bundle-size branch from 29d80e9 to 4551f90 Compare July 21, 2021 22:00

jimmywarting mentioned this pull request Aug 9, 2021

formdata shim denoland/node_shims#7

Closed

wojpawlik reviewed Aug 17, 2021

View reviewed changes

package.json Outdated Show resolved Hide resolved

wojpawlik reviewed Aug 18, 2021

View reviewed changes

package.json Outdated Show resolved Hide resolved

MattiasBuelens changed the title ~~Reduce bundle size~~ Rework variants to reduce bundle size Aug 18, 2021

wojpawlik approved these changes Aug 19, 2021

View reviewed changes

package.json Outdated Show resolved Hide resolved

jimmywarting mentioned this pull request Aug 23, 2021

v3 Roadmap node-fetch/node-fetch#668

Closed

35 tasks

MattiasBuelens added 12 commits September 6, 2021 00:22

Make ponyfill the default variant

1a4488c

Fix missing exported types

567ec37

Update readme

92cab9b

Remove "browser" field in package.json

e6fb1ef

"browser" is unnecessary if it's equal to "main".

Add polyfill.d.ts for global augmentations

1200812

Require TypeScript 3.6+ for type definitions

443e389

Move dist/types/ to types/

0999b8f

Pin to TypeScript 4.3 for now

201166a

Blocked by rollup/plugins#983

Update dependencies

e270ad8

Update readme

ccd9389

Fix heading levels

58e5f57

Add migration guide

d7f5595

MattiasBuelens force-pushed the reduce-bundle-size branch from 2b124c4 to d7f5595 Compare September 5, 2021 22:26

MattiasBuelens marked this pull request as ready for review September 5, 2021 22:42

MattiasBuelens mentioned this pull request Sep 6, 2021

ERR_UNSUPPORTED_DIR_IMPORT #87

Closed

MattiasBuelens added 2 commits September 6, 2021 22:04

Update changelog [skip ci]

762f8d9

Update link in readme [skip ci]

3e1d1e8

MattiasBuelens force-pushed the reduce-bundle-size branch from 5dadd34 to 3e1d1e8 Compare September 6, 2021 20:04

MattiasBuelens merged commit 2c9c5f3 into next Sep 6, 2021

MattiasBuelens added this to the v4.0.0 milestone Sep 6, 2021

MattiasBuelens mentioned this pull request Sep 6, 2021

Support importing as ESM in Node #51

Closed

1 task

MattiasBuelens deleted the reduce-bundle-size branch September 6, 2021 20:21

MattiasBuelens mentioned this pull request Sep 7, 2021

fix(blob-stream): Bump major web stream polyfill v4 node-fetch/fetch-blob#116

Closed

5 tasks

jimmywarting mentioned this pull request Oct 29, 2021

Reduce the size of dependencies node-fetch/node-fetch#1354

Closed

MattiasBuelens mentioned this pull request Nov 19, 2021

Add fallback for package exports #107

Merged

This was referenced Jan 4, 2024

Convert from/to native streams? #1

Closed

Rework variants to reduce bundle size #139

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework variants to reduce bundle size #83

Rework variants to reduce bundle size #83

MattiasBuelens commented Jul 21, 2021 •

edited

Loading

MattiasBuelens commented Jul 21, 2021

jimmywarting commented Jul 21, 2021

MattiasBuelens commented Jul 21, 2021

MattiasBuelens commented Aug 10, 2021 •

edited

Loading

jimmywarting commented Aug 11, 2021 •

edited

Loading

wojpawlik commented Aug 12, 2021

wojpawlik commented Aug 12, 2021

jimmywarting commented Aug 12, 2021

wojpawlik commented Aug 14, 2021

MattiasBuelens commented Aug 15, 2021

jimmywarting commented Aug 15, 2021 •

edited

Loading

jimmywarting commented Aug 15, 2021

MattiasBuelens commented Aug 15, 2021 •

edited

Loading

jimmywarting commented Aug 15, 2021

MattiasBuelens commented Aug 16, 2021

wojpawlik commented Aug 17, 2021

MattiasBuelens commented Aug 18, 2021

MattiasBuelens commented Aug 18, 2021

MattiasBuelens commented Aug 19, 2021

MattiasBuelens commented Aug 27, 2021

MattiasBuelens commented Sep 6, 2021

jimmywarting commented Sep 6, 2021 •

edited

Loading

jimmywarting commented Sep 6, 2021

Rework variants to reduce bundle size #83

Rework variants to reduce bundle size #83

Conversation

MattiasBuelens commented Jul 21, 2021 • edited Loading

MattiasBuelens commented Jul 21, 2021

jimmywarting commented Jul 21, 2021

MattiasBuelens commented Jul 21, 2021

MattiasBuelens commented Aug 10, 2021 • edited Loading

jimmywarting commented Aug 11, 2021 • edited Loading

wojpawlik commented Aug 12, 2021

wojpawlik commented Aug 12, 2021

jimmywarting commented Aug 12, 2021

wojpawlik commented Aug 14, 2021

MattiasBuelens commented Aug 15, 2021

jimmywarting commented Aug 15, 2021 • edited Loading

jimmywarting commented Aug 15, 2021

MattiasBuelens commented Aug 15, 2021 • edited Loading

jimmywarting commented Aug 15, 2021

MattiasBuelens commented Aug 16, 2021

wojpawlik commented Aug 17, 2021

MattiasBuelens commented Aug 18, 2021

MattiasBuelens commented Aug 18, 2021

MattiasBuelens commented Aug 19, 2021

MattiasBuelens commented Aug 27, 2021

MattiasBuelens commented Sep 6, 2021

jimmywarting commented Sep 6, 2021 • edited Loading

jimmywarting commented Sep 6, 2021

MattiasBuelens commented Jul 21, 2021 •

edited

Loading

MattiasBuelens commented Aug 10, 2021 •

edited

Loading

jimmywarting commented Aug 11, 2021 •

edited

Loading

jimmywarting commented Aug 15, 2021 •

edited

Loading

MattiasBuelens commented Aug 15, 2021 •

edited

Loading

jimmywarting commented Sep 6, 2021 •

edited

Loading