investigate slow DTS build

[holic]

• do we need this in dev environments? or just before publishing packages?
• why is it so slow and resource heavy? is it our complex, old config types?

currently store-sync is eating up ram and causing oom issues in some enviroments

[holic]

cc @tash-2s in case you have some input/guidance

[tash-2s]

Some context: pnpm tsup in store-sync might fail on some machines.

To reproduce the error:

$ NODE_OPTIONS=--max-old-space-size=2048 pnpm tsup
CLI Building entry: src/index.ts, src/sqlite/index.ts, src/postgres/index.ts, src/postgres-decoded/index.t
s, src/recs/index.ts, src/trpc-indexer/index.ts, src/indexer-client/index.ts, src/zustand/index.ts
CLI Using tsconfig: tsconfig.json
CLI tsup v6.7.0
...
ESM ⚡️ Build success in 25ms
DTS Build start
node:events:495
      throw er; // Unhandled 'error' event
      ^

Error [ERR_WORKER_OUT_OF_MEMORY]: Worker terminated due to reaching memory limit: JS heap out of memory
    at new NodeError (node:internal/errors:405:5)
    at [kOnExit] (node:internal/worker:301:26)
    at Worker.<computed>.onexit (node:internal/worker:222:20)
Emitted 'error' event on Worker instance at:
    at [kOnExit] (node:internal/worker:301:12)
    at Worker.<computed>.onexit (node:internal/worker:222:20) {
  code: 'ERR_WORKER_OUT_OF_MEMORY'
}

Node.js v18.19.0

---

do we need this in dev environments? or just before publishing packages?

We don't need .d.ts files locally with our current setup, so we can conditionally turn off the dts flag.

I believe most TypeScript packages should have .js and .d.ts, and then they can optionally add .ts and .d.ts.map for devex. (See also: #2051)

So, here's a possible way to avoid the memory issue for now:

1. Run TSUP_ENV=development pnpm tsup for local environments.
2. In tsup.config.ts, turn off dts by checking process.env.TSUP_ENV.

why is it so slow and resource heavy?

I've also noticed that the tsup command is unusually slow in store-sync. It seems tsc currently fails in store-sync on the main branch intentionally because it's actively under development. Generating .d.ts files indeed adds some cost, but the current slowness might also relate to the failing types, but not sure now. I wouldn't be surprised if it can't build .d.ts files at all in that condition (failing tsc).

---

Regarding the NODE_OPTIONS option, if your machine has sufficient available memory, the default value should be 4096, in which case the process should complete successfully.

So, another potential workaround could be to use NODE_OPTIONS=--max-old-space-size=3072 pnpm tsup if you encounter the error, but it may be painfully slow if it uses swap.

[tash-2s]

Also, another point about "slow DTS build" is that if the tsc runs slowly, the DTS build tends to be slow as well. For the JS build, we use esbuild internally, which just strips TS types, so it's fast.