Scanner.js start-up performance could be improved

[justjake]

Hello y'all! I ran some profiles of Notion's startup performance today and noticed that Linkifyjs is very expensive to import. Specifically, the file linkifyjs/lib/linkify-core/scanner.js file appears to be quite expensive. I wonder if the tactic of using "a|b|c|...".split('|') is expensive, or if the creation of the parser states involves some kind of n^2 computation.

Anyways, I was surprised to find that Linkify was the single most expensive dependency that Notion imports, and I wanted to let you know in case you find the information useful!

Here's a screenshot from Chrome DevTools of the boot up of the Notion web-app in a production-like environment. You can see how importing linkify compares to some other 3rd party dependencies we use like Moment and Prism.js.

My suggestion is that you take some time to profile the scanner, and see what work you can defer to the first run of a public method, so the library is lazy internally.

[nfrasser]

Hi @justjake, thanks for reporting this, I'll be looking at this over the next couple of days

[nfrasser]

Narrowed down the performance hit to the construction of the state machine for handling the 1500+ TLDs: https://github.com/Soapbox/linkifyjs/blob/a565b6fa8ad095f6f3e453abe3fdfa7221040653/tlds.js

Emptying out the list narrows down the initialization time from ~32ms to ~2ms on my machine (converting from .split() to an array did not significantly improve performance)

Going to explore a few options to address this:

• Precompute the state machine at build time (with a likely significant increase in file size)
• Take out most of the TLDs and leave out only the most common ones like .com, .net, etc. with an optional "extended TLDs" plugin (breaking change)
• Some other to-be-determined computational trick to speed up that code

[nfrasser]

• Delay the initialization until Linkify is called for the first time

[nfrasser]

Good news, managed a combination of all of the above to get the initial load time down from ~30ms to ~1ms. Managed this with both revamped internals + delayed initialization. The means there's an extra penalty you now pay the first time you run linkify (only about ~4ms thanks to the revamped internals), but future runs are faster than ever.

The fix is now available in v3.0.0-beta.1. A production release will be out in the coming weeks.

[justjake]

Amazing!