Increase the number of blocks to keep stale messages

[MechanicalTyler]

Related Github tickets

• Block Time: Reduce Paloma Block time to 1 second by lowering the commit timeout to 1 second VolumeFi/paloma#630

Background

We just increase our block speed drastically. As such, we need to increase the amount of blocks to allow messages to remain stale. This is because we have more blocks per minute.

We also don't need to check quite as often for the same reason.

Testing completed

• test coverage exists or has been added/updated
• tested in a private testnet

Breaking changes

• I have checked my code for breaking changes
• If there are breaking changes, there is a supporting migration.

[byte-bandit]

I would also be interested in performance metrics in general. There's a lot of stuff happening on every block, especially I see a lot of unoptimised/unbatched kv store reads/writes, and we just upped execution speed by 500% ish.

[taariq]

@byte-bandit any recommendations for monitoring and evaluating call performance?

[byte-bandit]

@taariq Traditionally I'd use some tracing in each modules end block logic, with that telemetry being sent and processed on services like Datadog or Grafana.

With a decentralised system, maybe a monitoring module is the more idiomatic approach. Something that'll hook into the end block logic and track time, storing the results with each block on the chain so they're retrievable by anyone.

However, since anyone can run a node and we have little to no information what kind of hardware each node is running on, I don't think those results will be too reliable. Reproducible profiling as part of CI benchmarking would be better, but that's something for a full time devops role.

[taariq]

We will wait for the screams of block signing and jailing agony, then. Merging!