Blocktree now only loads complete blocks (err, slots)

[mvines]

Loading a Blocktree at node startup is tedious because the code needs to care about a last_entry_id, as the ledger may halt at any entry. But once the node is running it never needs to care about a last_entry_id again as rotation always occurs on a last_id boundary (eg, bank.last_id()).

By restricting Blocktree to only load complete blocks, we can remove the usage of last_entry_id from all ledger processing at fullnode startup

Note that at present the same effect could have been accomplished by halting ledger loading at the last tick of a partially filled block. However this approach would break when we move from a tick-based last_ids to a block-based last_ids.

TODO:

• Land Populate test ledgers with a full slots to reduce test boilerplate #2963
• Wait for blocktree_processor to use slots as bank ids and squash() #2949 to land, rebase on it
• Fix all broken ledger-based tests

[rob-solana]

where's the proposal to change last_id to a block's last_id?

[mvines]

#2710

[carllin]

@mvines, why do we want to delete a partial slot, rather than just stopping processing at the end of the previous slot and letting replay stage detect the forks?

[mvines]

I’d love to not delete anything. Hmmm, yeah I think you’re probably right. If startup processing just stops at the last fully completed slot, replay stage should be able to read the partial last slot and pick up from there. I’ll give that a shot

[rob-solana]

this change will eliminate a couple of parameters to replay_stage startup, but is the main goal simplification of TPU startup?

[mvines]

Main goal is to make most of the code base stop caring about last_entry_id, which definitely includes both TPU and TVU startup.

[codecov]

Codecov Report

Merging #2928 into master will increase coverage by <.1%.
The diff coverage is 97%.

@@           Coverage Diff            @@
##           master   #2928     +/-   ##
========================================
+ Coverage    76.7%   76.8%   +<.1%     
========================================
  Files         130     130             
  Lines       19993   20039     +46     
========================================
+ Hits        15352   15403     +51     
+ Misses       4641    4636      -5

[rob-solana]

a TPU will slash itself if restarted during its slot, I think?

[mvines]

We might want to add some logic later where if a node that tries to restart as slot leader skips it's slot entirely and lets the cluster move to the next leader (well once we can do that!)