Benchmark

[sindresorhus]

Re https://twitter.com/sindresorhus/status/689851060303872000

We should set up a benchmark with various types of test suites, like one with sync tests in one file, one with multiple files and async tests. This to make sure we don't regress in performance and ability to compare ourselves to other test runners.

[kentcdodds]

Thanks for taking this so seriously! 😄 I'm currently working on a repo to compare mocha and ava with a lot of tests with a lot of files. I'm no benchmark expert, but maybe this will be helpful. Will comment when I've pushed it up. Thanks!

[jfmengels]

I have created this repo for the same purpose: https://github.com/jfmengels/mocha-vs-ava-performance
It's pretty small, but it makes it obvious how big the difference is between Mocha's instant feedback in watch mode, and Ava's "watch" with Nodemon, or even single runs.

[jamestalmage]

watch mode

AVA is at an extreme disadvantage in watch mode until we implement our own watch mechanism. That's certainly on the roadmap, but we've been punting on it in favor of other improvements. Not sure if it's time to tackle that yet or not.

[jamestalmage]

I have confirmed this commit caused a major performance hit. I think I've identified the problem and I am working up a solution.

See twada/empower-core#7

[twada]

Just released empower-core 0.5.0 🎉 (Thanks @jamestalmage !!)

[sindresorhus]

@jamestalmage did a superb job adding a pretty benchmark in #496. It's easy to add additional things to benchmark, so suggestions welcome.

[sindresorhus]

Instructions here: c22eabd