Multithreaded vote processor

[pwojcikdev]

Changes vote processor to use multiple threads. While processing of votes inside the election container is still done under exclusive lock, this should help amortize time spent checking vote signatures.

[gr0vity-dev]

I tested this on a local network, and somehow the multi-threaded approach behaves inferior to the current develop branch.

Here are the grafana stats

[pwojcikdev]

That's interesting, thanks for testing @gr0vity-dev. This would imply that the overhead of lock contention is the largest factor here. I fixed a small bug with thread notifications, but I don't think it should have a significant impact. It would be great if you could run your tests with various settings for batch_size and threads. For now I modified the default config to use a single thread by default.

[clemahieu]

A frame graph would be good at identifying what's causing the slowdown, seems odd that one thread would be the fastest for an explicitly CPU-bound task.

[pwojcikdev]

I received the flamegraph and was able to identify the bottleneck. It's now fixed and multithreaded version seems to perform better than single threaded, as expected.

[gr0vity-dev]

I did another side by side run between develop and the improved multi-threaded vote processor

Here are the detailed grafana stats