Browser support proposal

[tenbits]

Hi Team,

what do you think about browser compatibility?

TextMate grammars are really great. And it was a pity Highlights can't be used in browsers, as oniguruma native regex library is being used. So I decided, to invest some time to implement oniguruma compatible javascript regex engine - Regex. It uses JavaScript's RegExp, and not supported features are then handled seperatly. Afterwards, I have created node-oniguruma-compatible module - oniguruma-js. All the unit tests from the node-oniguruma are being passed. And as the final point, a fork of first-mate, which uses the oniguruma-js as the dependency, passes also all the tests. The benchmark/large.js and benchmark/bootstrap.css are tokenized the same - as with node-oniguruma.

Performance

As for know, it is slower then node-oniguruma. But node-oniguruma should not be completely replaced with the javascript implementation, only in environments, where native modules can't be used - like browsers

with oniguruma-js

Generated 8830 tokens for 8830 lines in 1859ms (5 tokens/ms)
Generated 7 tokens for 7 lines in 6473ms (0 tokens/ms)
Generated 5786 tokens for 5786 lines in 756ms (8 tokens/ms)
Generated 7 tokens for 7 lines in 5023ms (0 tokens/ms)

with node-oniguruma

Generated 8830 tokens for 8830 lines in 1162ms (8 tokens/ms)
Generated 7 tokens for 7 lines in 1425ms (0 tokens/ms)
Generated 5786 tokens for 5786 lines in 515ms (11 tokens/ms)
Generated 7 tokens for 7 lines in 1047ms (0 tokens/ms)

The performance for the minified code is absolutely bad, as the lines are extremely long in that case. It will be much better, when browers support sticky flags natively. oniguruma-js is the first draft - there are a lot of things to optimize, fix and re-factor.

If you are interested to bring highlights and first-mate to browsers some code refactors are then also required. For example, the IO things should be encapsulated into a Grammar Manager - to support bundling and different data transports. Dependencies should be reduced. And so on.

What do you think all about this?
Cheers, Alex

[formula1]

This is super sweet! I'm not part of the team but +1

[chenglou]

The native dependency is making some of the work hard for us too. Would love to see the switch!