Add a jmh benchmark for WorldStateDownloader #972
Conversation
| final WorldStateArchive worldStateArchive = new WorldStateArchive(storage); | ||
| final MutableWorldState worldState = worldStateArchive.getMutable(); | ||
|
|
||
| dataGen.createRandomAccounts(worldState, 10000); |
There was a problem hiding this comment.
Is JMH the right place to try larger states? It wold be interesting to see if the performance decline is linear or exponential, or worse, so we could do world states at 1,000, 10,000, and 100,000, and 1,000,000 to see what the curve is.
There was a problem hiding this comment.
JMH turned out to be a fairly poor fit for this type of test - we could parameterise it so we could generate larger states but the runtime is already pretty brutal because we wind up having to recreate that state for every benchmark run and JMH wants to do a lot of runs to check consistency. This will give us some rough numbers to work off for now but to really drill into details a controlled network benchmark running the full node and a single 10-20 minute sync will give us something we can drill into that kind of detail on.
That said, in looking at Rinkeby fast syncs, the rate that we complete nodes is very consistent regardless of how many we've imported already. So it looks to be linear based on the number of nodes.
PR description
The benchmark is higher level than a JMH benchmark really should be and has to use the generally inadvisable
@Setup(Level.Invocation)to ensure that we start from a clean world state on each run. Otherwise the downloader detects the state is already downloaded and completes immediately. Given the benchmark takes multiple seconds the inaccuracy of measurement introduced byLevel.Invocationdoesn't affect results too much.Doing a full fast sync using a few full peers would give better data, but this provides some feedback in the meantime.