Investigate performance issue with esbuild.transformSync in bun

[Jarred-Sumner]

bench/snippets/transpiler.mjs in Bun:

❯ bun transpiler.mjs
cpu: Apple M1 Max
runtime: bun 0.5.1 (arm64-darwin)

benchmark      time (avg)             (min … max)       p75       p99      p995
------------------------------------------------- -----------------------------
• transformSync (11 KB jsx file)
------------------------------------------------- -----------------------------
bun        120.75 µs/iter   (116.46 µs … 1.96 ms) 120.38 µs  133.5 µs 143.29 µs
esbuild      4.53 ms/iter     (3.98 ms … 7.76 ms)   4.79 ms   6.58 ms   7.76 ms
swc        531.12 µs/iter   (389.21 µs … 2.98 ms) 539.96 µs   1.87 ms   2.35 ms
babel        3.97 ms/iter    (2.94 ms … 11.91 ms)   3.91 ms   9.75 ms  11.91 ms

summary for transformSync (11 KB jsx file)
  bun
   4.4x faster than swc
   32.88x faster than babel
   37.52x faster than esbuild

• tranform x 5
------------------------------------------------- -----------------------------
bun        252.26 µs/iter   (195.04 µs … 3.72 ms) 248.63 µs   1.08 ms   1.18 ms
esbuild      2.39 ms/iter     (1.66 ms … 9.04 ms)   2.33 ms   7.28 ms   8.47 ms
swc        968.48 µs/iter   (682.96 µs … 7.73 ms) 909.67 µs    3.1 ms   3.65 ms
babel       18.25 ms/iter   (14.84 ms … 24.66 ms)  19.55 ms  24.66 ms  24.66 ms

summary for tranform x 5
  bun
   3.84x faster than swc
   9.47x faster than esbuild
   72.35x faster than babel

In Node.js:

❯ node transpiler.mjs
cpu: Apple M1 Max
runtime: node v19.4.0 (arm64-darwin)

benchmark      time (avg)             (min … max)       p75       p99      p995
------------------------------------------------- -----------------------------
• transformSync (11 KB jsx file)
------------------------------------------------- -----------------------------
esbuild    832.48 µs/iter    (634.38 µs … 1.5 ms) 931.33 µs   1.33 ms   1.38 ms
swc        633.11 µs/iter    (402.5 µs … 4.69 ms) 613.67 µs   3.27 ms   4.26 ms
babel        4.25 ms/iter    (3.19 ms … 14.49 ms)   4.39 ms   8.49 ms  14.49 ms

summary for transformSync (11 KB jsx file)
  swc
   1.31x faster than esbuild
   6.72x faster than babel

• tranform x 5
------------------------------------------------- -----------------------------
esbuild      2.64 ms/iter     (1.83 ms … 9.26 ms)   2.59 ms   7.49 ms   7.57 ms
swc          1.39 ms/iter   (973.42 µs … 9.29 ms)   1.36 ms   5.34 ms   7.22 ms
babel       19.35 ms/iter   (17.49 ms … 30.68 ms)   19.1 ms  30.68 ms  30.68 ms

summary for tranform x 5
  swc
   1.89x faster than esbuild
   13.87x faster than babel

[Jarred-Sumner]

cc @evanw (not asking you to do any work)

[evanw]

Do you implement worker_threads and SharedArrayBuffer and atomics? See evanw/esbuild#590. Also esbuild has a version check to avoid buggy node versions: https://github.com/evanw/esbuild/blob/main/lib/npm/node.ts#L28-L41. If esbuild doesn’t use worker_threads then transformSync will be very slow indeed as each call will be a new child process.

[Jarred-Sumner]

Do you implement worker_threads

Not yet. That explains it.

[Jarred-Sumner]

Fixed by @dylan-conway in #901

Speeds are roughly the same, though the p99 implies Bun is cloning some data an extra time unnecessarily.

❯ node transpiler.mjs
cpu: AMD Ryzen 5 3600 6-Core Processor
runtime: node v20.5.0 (x64-linux)

benchmark      time (avg)             (min … max)       p75       p99      p995
------------------------------------------------- -----------------------------
• transformSync (11 KB jsx file)
------------------------------------------------- -----------------------------
esbuild      1.87 ms/iter     (1.33 ms … 2.47 ms)   1.98 ms   2.42 ms   2.45 ms

❯ bun transpiler.mjs
cpu: AMD Ryzen 5 3600 6-Core Processor
runtime: bun 0.7.2 (x64-linux)

benchmark      time (avg)             (min … max)       p75       p99      p995
------------------------------------------------- -----------------------------
• transformSync (11 KB jsx file)
------------------------------------------------- -----------------------------
esbuild      1.83 ms/iter     (1.16 ms … 3.12 ms)   2.03 ms   2.86 ms   3.05 ms