approval-distribution: Continue processing messages while waiting aft…

…er approval-voting

In, the current implementation every time we process an assignment or an
approval that needs checking after the approval voting, we will wait till
approval-voting answers the message.

Given that approval-voting will execute some signatures checks that take
significant time(between 400us and 1 millis) per message, that's where most of
the time in the approval-distribution, see
https://github.com/paritytech/polkadot/issues/6608#issuecomment-1590942235 for
the numbers.

So, modify approval-distribution, so that it picks another message from the
queue while the approval-voting is busy doing it's work.

This will have a few benefits:
1. Better pipelinening of the messages, approval-voting will always have work to
do and it won't have to wait for the approval-distribution to send it a message.
Additionally, some of the works of the approval-distribution will be executed in
parallel with work in approval-voting instead of serially.
2. By allowing approval-distribution to process messages from it's queue while
approval-voting confirms that a message is valid we give the
approval-distribution the ability to build a better view about what messages
other peers already know, so it won't decide to gossip messages to some of it's
peers once we confirm that message as being correct.
3. It opens the door for other optimizations in approval-voting subsystem, which
would still be the bottleneck.

Note!
I still expect the amount of work the combo of this two systems can do, to still
be bounded by the numbers of signatures checks it has to do, so we would have to
stack this with other optimizations we have in the queue.
- https://github.com/paritytech/polkadot/issues/6608
- https://github.com/paritytech/polkadot/issues/6831

[] Evaluate impact in versi
[] Cleanup code an make CI happy to make the PR meargeable.

Signed-off-by: Alexandru Gheorghe <[email protected]>