feat(cli): add a RPC batch option to cli

[0x-shanks]

This PR adds a RPC batch size option to the deploy command.

Resolve: #2236

[changeset-bot]

🦋 Changeset detected

Latest commit: c812110

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages

Name	Type
@latticexyz/cli	Major
@latticexyz/abi-ts	Major
@latticexyz/block-logs-stream	Major
@latticexyz/common	Major
@latticexyz/config	Major
create-mud	Major
@latticexyz/dev-tools	Major
@latticexyz/ecs-browser	Major
@latticexyz/faucet	Major
@latticexyz/gas-report	Major
@latticexyz/network	Major
@latticexyz/noise	Major
@latticexyz/phaserx	Major
@latticexyz/protocol-parser	Major
@latticexyz/react	Major
@latticexyz/recs	Major
@latticexyz/schema-type	Major
@latticexyz/services	Major
@latticexyz/solecs	Major
solhint-config-mud	Major
solhint-plugin-mud	Major
@latticexyz/std-client	Major
@latticexyz/std-contracts	Major
@latticexyz/store-cache	Major
@latticexyz/store-indexer	Major
@latticexyz/store-sync	Major
@latticexyz/store	Major
@latticexyz/utils	Major
@latticexyz/world-modules	Major
@latticexyz/world	Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[0x-shanks]

@holic

#2236 (comment)

There was no corresponding throttling, but there was one that did batch processing instead, so this was substituted.
https://viem.sh/docs/clients/transports/http#batch-json-rpc

In my environment, setting batchSize to 1000 allowed me to deploy without any errors.

I have closed PRs I made in the past and made new ones

#2237
↓
#2322

[holic]

Out of curiosity, does just using batch: true do the same thing for you? If so, I think we could feasibly turn this on by default for the deployer (assuming viem gracefully falls back if batching isn't supported for a particular RPC).

[0x-shanks]

I don't think just batch:true will work, because if you don't specify batch.wait, the batch control will not work for 1 second(default wait 0 second).
https://viem.sh/docs/clients/transports/http#batchwait-optional

[0x-shanks]

Do you usually update the document side with the same PR?

[holic]

If you read further up, it'll still batch within the same "tick" (see also zero delay):

This makes me suspect most of our operations would be batched because they're often sent simultaneously.

[holic]

Do you usually update the document side with the same PR?

Ideally, yep!

[0x-shanks]

The same effect as throttling could only be achieved if a sufficiently large batchSize and a sufficiently long wait were set

For example, each ensureHoge calls Promise.all internally, and Promise.all does multiple etc_calls as one step batch call
https://github.com/latticexyz/mud/blob/main/packages/cli/src/deploy/deploy.ts#L93-L114

It is thought that each step that is summarized in 1 Batch is a step that was executed within a certain period of time (within the time of waiting).

My hypothesis is this.

1. If batchSize is small, more batches are split, more requests are sent per second, and quote is limited.
2. If the wait time is short, requests are sent at short intervals even if batchSize is large, and quote is limited.

I am not sure what should be taken as the argument of cli

[0x-shanks]

I'm guessing based on the actual results I've tried, so I could be wrong.

[holic]

I'm guessing based on the actual results I've tried, so I could be wrong.

Could you try with just batch: true and let us know if this still hits RPC rate limits?

[0x-shanks]

Of course, I've tried batch: true and still hit RPC rate limits

My environment is this.

• 3 systems
• 20 tables
• 1 module
• RPC Endpoint: Alchemy Growth Plan (https://www.alchemy.com/pricing)

---

Here are the results of the search with each parameter fixed

• batchSize: 10, wait: 1000(ms): hit RPC rate limits
• batchSize: 100, wait: 1000(ms): no error
• batchSize: 1000, wait: 1000(ms): no error
• batchSize: 100, wait: 0(ms): hit RPC rate limits
• batchSize: 100, wait: 10(ms): hit RPC rate limits
• batchSize: 100, wait: 100(ms): hit RPC rate limits

Since batchSize and wait are defaults when batch: true, respectively
batchSize: 1000, wait: 0(ms)

[holic]

asked if we could enable this by default but sounds like there's no good way to detect if batching is supported so keeping it as an option makes sense: https://discord.com/channels/1156791276818157609/1212120089219043369