Performance testing of PoA networks

[rstormsf]

Abstract

In searching of a perfect PoA setup, we, as software engineers, would need to benchmark different configurations of AuRa setup. One of the few examples would be to find the maximum number of validators under which the AuRa can still perform well. Another example, is decreasing block times to find the optimal balance between a secure network and a fast network.

Rationale

Cases that we need to perform the tests:

• Deploy and install a PoA network to find a maximum boundary number of validators managed by consensus smart contracts having the same 5 second block times
• Find the minimum possible block time that AuRa can provide, preserving the same security rules(double spending, time sync attack, etc..)

Specification

As a result of such tests, the community would need to look at a detailed report with different configuration tests and benchmarks.

Implementation

Only one Parity node is allowed per cloud instance node.

Tests with heavy loads of DDoS transactions needs to be implemented and executed against this network.

[phahulin]

Few more things

• max possible block gas limit => max number of txs/block that nodes will still be able to process
• maybe simulate network issues (what will happen in a network with many validators if new blocks are not propagating fast enough)
• for AuRa maybe test time-sync issues

Network tests need to be performed for different consensus algorithms, if they are available.

Also I think it's important that these tests be performed while network is under constant load of newly established & dropped external connections. As an illustration, on sokol, there are approx. 5-10 new connections to a node per second. Most of them are "spam" (incorrect genesis, incorrect clients like geth, mist, ..), so they are dropped quickly. However, I think they influence stability of connection between real nodes (reason have to use reserved_peers over bootnodes)

[vbaranov]

Also, we have a helpful tool for deployment of POA Network: https://github.com/poanetwork/poa-test-setup. It might be helpful, for those, who will be making performance testing. POA Network can be deployed with a couple of NPM scripts on a local machine. Basic scenarios of deployment are listed in README.

[igorbarinov]

The configurations for performance tests:

• validators required [15, 25], later will have 100 validators
• block time [5s, 3], later will have 1s
• GasLimit [8m, 80m]
• engine [aura, tendermint]

Load profile:

• tx load [acc2acc, contract+tx2contract+acc2acc]

[rstormsf]

First part would be write efficient script to spam a network with thousands tx per second.
Things to consider:

• tx nonce for every account, that means you have to send 10,000 tx from all 10,000 different accounts
• efficient language (rust, elixir, etc.)
• efficient db storage that can support concurrent db read/write