Replace fork-ts-checker-webpack-plugin with faster alternative

[Timer]

This removes fork-ts-checker-webpack-plugin and instead directly calls the TypeScript API.

This is approximately 10x faster.

Base build: 7s (no TypeScript features enabled)

• [email protected]: 90s, computer sounds like an airplane
• [email protected]: 84s, computer did not sound like an airplane
• [email protected]: 90s, regressed
• npx tsc -p tsconfig.json --noEmit: 12s (time: 18.57s user 0.97s system 169% cpu 11.525 total)
• This PR: 22s, expected to get better when we run this as a side-car

All of these tests were run 3 times and repeat-accurate within +/- 0.5s.

[johnnyreilly]

You will no doubt will (and should!) pick the approach that works best for you. Couple of things to note in case there are properties of the fork-ts-checker-webpack-plugin that are directly useful to the project (eg the "webpack awareness" / support for eslint etc):

• it looks like you're using an old version (V3?) of the checker - I wonder how v4 compares?
• related to that @piotr-oles has been doing some incredible work for v5 Rewrite to version 5.0.0 TypeStrong/fork-ts-checker-webpack-plugin#404 - even though it's technically alpha it's pretty well advanced and I'd recommend taking it for a spin!

Of course, please make the calls that work for your project, I just wanted to let you know some possibilities 🤗

[Timer]

@johnnyreilly Thanks for reaching out!

We were pretty happy with the plugin overall, but there's a few things other than speed that motivated this change:

• We do not run type checking in development, only in production (so we use sync mode, not async, and no watching).
• We want to run type checking before the webpack build, and skip it [webpack] if checks fail. This gives a faster turnaround time IMO.
• We firmly believe in keeping the webpack build's responsibilities as small as possible. We're going to start moving as many things as we can into side-cars, completely off the Node.js event loop (this PR is the first step).

Thanks for mentioning that newer versions may have seen speed improvements!
Since this work is already done, we'll probably stick with it.

To elaborate more on the speed we were observing:
next build + fork-ts-checker-webpack-plugin = 86s
next build with no TS checks = 6s
tsc -p tsconfig.json = 10s

I may have some spare time tomorrow to see what the speed is like on the new version(s).

[johnnyreilly]

Thanks for the detailed response @Timer! It does sound like you're planning some pretty hefty (and cool sounding!) architectural changes. Super interesting that you don't run type checking in development - that's where I particularly appreciate it!

We want to run type checking before the webpack build, and skip it [webpack] if checks fail. This gives a faster turnaround time IMO.

This is a really interesting idea. I'm very curious as to how it works out.

We firmly believe in keeping the webpack build's responsibilities as small as possible. We're going to start moving as many things as we can into side-cars, completely off the Node.js event loop (this PR is the first step).

Super interesting - as the name suggests, the initial motivation for fork-ts-checker-webpack-plugin was exactly that: "Webpack plugin that runs TypeScript type checker on a separate process."

Sounds like you're going all in on this. Fascinating.

If you do get a chance to compare v4 and v5 alpha performance we'd love to get an idea of the numbers.

Thanks again for sharing the ideas here - really appreciate it!

[Timer]

@johnnyreilly I just ran some fresh tests on the latest version of our project:

Base build: 7s (no TypeScript features enabled)

• [email protected]: 90s, computer sounds like an airplane
• [email protected]: 84s, computer did not sound like an airplane
• [email protected]: 90s, regressed
• npx tsc -p tsconfig.json --noEmit: 12s (time: 18.57s user 0.97s system 169% cpu 11.525 total)
• This PR: 22s, expected to get better when we run this as a side-car

All of these tests were run 3 times and repeat-accurate within +/- 0.5s.

---

It seems v4 helped (and doesn't burn CPU), but v5 re-regressed. This PR is still beating the plugin by 60+ seconds.

Would love to go more in-depth sometime to figure out the performance bottleneck!

[johnnyreilly]

Thanks @Timer - that's tremendously helpful! @piotr-oles it looks like there's a performance degradation with v5 - see the numbers from @Timer above.

[Timer]

Thank you again for reaching out! ❤️ 🚀

---

Super interesting that you don't run type checking in development - that's where I particularly appreciate it!

Missed this bit!

We disabled it because users (and us) experienced slower dev-time compilations, despite using async: true. The plan is to re-introduce development time checks with this new setup!

I'd be happy to help/collab to figure out the upstream issue though so we don't have to maintain this ourselves. We're also exploring ESLint in the future, so this all parlays.

[johnnyreilly]

We're also exploring ESLint in the future too, so this all parlays.

Did you use the ESLint integration in fork-ts-checker-webpack-plugin BTW? My own use case is running fork-ts-checker-webpack-plugin to surface up both type checking errors from TypeScript and lints from ESLint.

[Timer]

We did not: strictly the TypeScript integration. Next.js doesn't (currently) ship with any form of ESLint (not even eslint-loader).

The DX we're aiming for (in dev) is that TS/ESLint errors show up in a toast though, and not a full overlay ("compile error").

---

For posterity, here was our config:

new _forkTsCheckerWebpackPlugin.default(
  _pnpWebpackPlugin.default.forkTsCheckerOptions({
    typescript: typeScriptPath,
    async: false,
    useTypescriptIncrementalApi: true,
    checkSyntacticErrors: true,
    tsconfig: tsConfigPath,
    reportFiles: ["**", "!**/__tests__/**", "!**/?(*.)(spec|test).*"],
    compilerOptions: { isolatedModules: true, noEmit: true },
    silent: true,
    formatter: "codeframe",
  })
);

Just tested: removing _pnpWebpackPlugin.default.forkTsCheckerOptions did not affect the time it took to run.

[johnnyreilly]

The DX we're aiming for (in dev) is that TS/ESLint errors show up in a toast though, and not a full overlay ("compile error").

Yeah that was exactly why I wrote https://github.com/johnnyreilly/fork-ts-checker-notifier-webpack-plugin

Though I wanted all lints / errors as notifications / toasts.

Something very lovely about having errors as toasts outside of the application. I've never really enjoyed the full overlay approach; I like to have the application running even if it does contain errors. I find value in being able to debug a broken application - particularly if it's a type system error which may not massively impair functionality.

[johnnyreilly]

Incidentally, I've an idea that isolatedModules: true may hurt your performance and be unnecessary now. Could be wrong about that - have an idea something changed around TypeScript 3.6 / 3.7.

skipLibCheck: true could improve your performance (regardless of context).

Incidentally, skipLibCheck: true is going to become the default as of TypeScript 3.9

https://www.twitter.com/orta/status/1260583041740812288 announced by @orta

[piotr-oles]

@Timer Thanks for the benchmark, I will try to investigate why there is such a big difference between tsc and fork-ts-checker-webpack-plugin since we use the same API 😃

[timneutkens]

skipLibCheck: true could improve your performance (regardless of context).

It's already true in the benchmarks @Timer did (so didn't affect the slowdown)

[piotr-oles]

@Timer Could you tell me how can I check timings on my local machine? I cloned the repository and checkout master branch but I'm not sure which command I should run to use fork-ts-checker-webpack-plugin :)

[Timer]

@piotr-oles the project we're testing with is a private one (that powers our website), happy to provide anything you need to see tho (minus full repo).

[piotr-oles]

@Timer Ok, I will try on different repository :) Could you tell me which version of the typescript did you use for tests?

[timneutkens]

    "typescript": "3.7.2"

[piotr-oles]

Thanks :) If someone has a project that can share and that is significantly slower with fork-ts-checker-webpack-plugin than with tsc, it would be very helpful for testing 😃

[piotr-oles]

I did some research regarding plugin performance and these are outcomes:

• the fork-ts-checker-webpack-plugin is slower than tsc partially because it runs in parallel with the webpack. I'm not sure about the cause of it but I would bet on the file system. It was about 20% slower in my case.
• there was also some difference even if the plugin ran without webpack compilation in parallel. I didn't found the cause (was about 15% slower).
• I improved the initial compilation time by using incremental: true by default. The time dropped from ~100 s to ~30 s (feat: improve overall typescript performance TypeStrong/fork-ts-checker-webpack-plugin#435)
• I fixed watcher implementation for incremental compilation. Thanks to that the incremental time dropped from ~15 s to ~2 s (feat: improve overall typescript performance TypeStrong/fork-ts-checker-webpack-plugin#435)

[timneutkens]

Awesome @piotr-oles 🚀