Explore: Remove dead sectors in a network upgrade

[arajasek]

User Story

I, a Filecoin participant, would like the size of a single state tree (and a single snapshot, etc.) to be smaller. One source of bloat in the state tree is expired SectorInfos. While there are considerations to be made before simply removing them all (eg. would we ever want to allow "reviving" dead sectors?), I would like to know the feasibility and impact of removing all dead sectors in a network upgrade.

Acceptance criteria

• Exact amount of state size that would be freed up if we ran this on mainnet today
• How long such a migration would take on an average machine (both premigration and migration)
• How many partitions (if any) would be removed in total as a result of this

Technical Breakdown

• Build a migration that runs the CompactPartitions logic for all partitions, on every deadline, for every miner
• Run this once, and record the change in state size as well as the time taken
• If worthwhile, re-run with a premigration and other optimizations

[beck-8]

Let me understand, if these data are deleted, will this interface be unusable? StateMinerSectors
If so, the scope of influence is still very wide, and some tools of lotus itself will rely on it, and personnel related to data analysis will rely on it.

[arajasek]

@beck-8 No, that interface should work as before -- it will just return less information, because the "dead" sectors (if any) will no longer be returned.

[beck-8]

I see what you mean, I'm talking about this situation. After deleting the data, in most cases will continue in StateMinerActiveSectors , I stand by my opinion, in programming, people who depend on this interface will need some time to adjust.

[jennijuju]

exploration ongoing here filecoin-project/FIPs#795 (comment)