Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main' into no-dupe-keys
Browse files Browse the repository at this point in the history
* upstream/main:
  fix(rome_js_analyze): Verify method name of React API calls (rome#3619)
  benchmark: Add pprettier and dprint to benchmark (rome#3597)
  feat(vscode): display the version of the language server in the status bar (rome#3616)
  fix(rome_cli): correctly account for diff diagnostics in the printed diagnostics count (rome#3595)
  fix(rome_cli): Respect formatter/linter `enabled` from configuration (rome#3591)
  Remove dead styles
  [website] Add scroll-padding-top
  • Loading branch information
jeysal committed Nov 9, 2022
2 parents 83702b5 + 76a56db commit 9fffd25
Show file tree
Hide file tree
Showing 38 changed files with 1,833 additions and 1,142 deletions.
17 changes: 12 additions & 5 deletions benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,17 @@ Setup: MacBook Pro (13-inch, M1, 2020)

### Formatting
* Rome's ~25 times faster than Prettier
* Rome's ~7 times faster when restricting it to a single core
* Rome's ~20 times faster than parallel-prettier
* Rome's ~20 times faster than `xargs -P`[^1]
* Rome's 1.5-2 times faster than `dprint`
* Rome single-threaded is ~7 times faster than Prettier.


[^1]: Run `time find lib/ examples declarations benchmark -name '*.js' -print0 | xargs -P8 -0 -n 200 npx prettier --write --loglevel=error` in the `target/webpack` directory. I manually tinkered with the `-n` parameter to get the fastest run.

### Linting
* Rome's 2-3 times faster than ESLint
* Rome is ~20-40% slower than ESLint when restricting it to a single core.
* Rome's ~15x times faster than ESLint
* Rome single-threaded is ~4 times faster than ESLint.

The speed-ups for the multithreaded benchmarks can vary significantly depending on the setup. For example, Rome is 100 times faster than Prettier on an M1 Max with 10 cores.

Expand All @@ -23,16 +29,17 @@ The speed-ups for the multithreaded benchmarks can vary significantly depending
* It should be possible to speed up Prettier. Rome's architecture isn't that different, and native has its advantages, but Prettier should be able to compete in single-threaded mode.

### Linting
* Rome's linter is fast but there is room for improvements
* Rome's linter spends significant time building the semantic model, the control flow graph, and matching queries. I'm convinced there's room for improvement ([3565](https://github.com/rome/tools/pull/3565), [3569](https://github.com/rome/tools/pull/3569)).
* Computing the diff for code fixes is expensive. Rome can spend up to 3s computing diffs (not measured by these benchmarks, see explanations below)
* Hypothesis: Rome doesn't use async IO to read files. That's why the single-threaded Rome issues the file read commands one by one. ESLint may be faster in single-threaded linting because it can issue all reads with async IO (so that the OS loads the files in the background while other files are linted). I have yet to verify if ESLint indeed does use async IO.

## Notes

We've been careful to create fair benchmarks. This section explains some of the decisions behind the benchmark setup and how these decisions affect the results. Please [let us know](https://github.com/rome/tools/issues) if you have ideas on how to make the benchmarks fairer or if there's a mistake with our setup.

### Formatting
* Compares the wall time of Rome and Prettier to format all files in a project where all files are correctly formatted.
* Compares the wall time of Rome, Prettier, and dprint to format all files in a project where all files are correctly formatted.
* dprint and Prettier support incremental formatting to only format changed files, whereas Rome does not. This benchmark does not measure incremental formatting as it measures cold formatting time. You may see significant speedups on subsequent formatting runs when enabling incremental formatting.
* The benchmark limits Prettier to only format JavaScript and TypeScript files because Rome doesn't support other file types.
* Rome only prints a summary with the number of formatted files. The prettier benchmark uses `--loglevel=error` for a fairer benchmark so that Prettier doesn't print every filename.

Expand Down
12 changes: 12 additions & 0 deletions benchmark/dprint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"incremental": false,
"typescript": {
},
"includes": ["**/*.{ts,tsx,js,jsx,cjs,mjs}"],
"excludes": [
"**/node_modules"
],
"plugins": [
"https://plugins.dprint.dev/typescript-0.77.0.wasm"
]
}
Loading

0 comments on commit 9fffd25

Please sign in to comment.