Stop panicking when pruning unused queued blocks

[teor2345]

Motivation

1. When Zebra prunes queued blocks, it drops their senders without sending a response. This caused a panic in one Zebra testnet node, shortly after NU5 activation.

2. When Zebra prunes queued blocks, it doesn't remove their known UTXOs.

This is unexpected work in Sprint 20.

Logs

The panic happens when the CommitBlock future checks for a response on its oneshot receiver.

Oct 06 22:10:10.960  INFO {zebrad="339fefb" net="Test"}:crawl_and_dial{crawl_new_peer_interval=60s}: zebra_network::peer_set::candidate_set: timeout waiting for the peer service to become rea
dy                                                                                                                                                                                             
Oct 06 22:10:28.037  INFO {zebrad="339fefb" net="Test"}:sync:obtain_tips: zebra_network::peer_set::set: network request with no ready peers: finding more peers, waiting for 13 peers to answer
 requests address_metrics=AddressMetrics { responded: 13, never_attempted_gossiped: 0, never_attempted_alternate: 0, failed: 546, attempt_pending: 49, recently_live: 13, recently_stopped_resp
onding: 0 }                                                                                                                                                                                    
Oct 06 22:10:41.680  INFO {zebrad="339fefb" net="Test"}:crawl_and_dial{crawl_new_peer_interval=60s}: zebra_network::peer_set::candidate_set: timeout waiting for the peer service to become rea
dy                                                                                                                                                                                             
The application panicked (crashed).                                                                                                                                                            
Message:  sender is not dropped: RecvError(())                                                                                                                                                 
Location: zebra-state/src/service.rs:694                                                                                                                                                       
                                                                                                                                                                                               
Metadata:                                                                                                                                                                                      
version: 1.0.0-alpha.17+39.g339fefb                                                                                                                                                            
Zcash network: Testnet                                                                                                                                                                         
state version: 10                                                                                                                                                                              
branch: main
git commit: 339fefb                            
commit timestamp: 2021-10-06T01:08:41+00:00
target triple: x86_64-unknown-linux-gnu
build profile: release

Solution

• Send an error on pruned queued blocks
• Remove known UTXOs from pruned queued blocks
  • Add tests for UTXO removal

Review

Anyone can review this PR.

Reviewer Checklist

• Code implements Specs and Designs
• Tests for Expected Behaviour
• Tests for Errors