Large scale testing post FIP13/FIP8

[Stebalien]

When FIP13 & FIP8 land, onboarding throughput should increase. We need to verify that lotus can handle this.

1. Estimate maximum onboarding rate given FIP13 & FIP8 and current gas limits.
2. Given this onboarding rate, estimate the number of on-chain sectors in 1, 2, 5 years.
3. Verify that lotus can support this number of sectors:
   1. Window PoSt messages can be submitted.
   2. Sector expiration/termination won't cause performance issues.
   3. Sector faults/missed window PoSt messages won't cause performance issues.
   4. State doesn't grow too large.

Outputs:

• Spreadsheet with estimates/calculations for chain size, timing, object size, maximum miner sizes, etc.
• A tool that creates a large state snapshot so we start a devnet from a "large" state.
• Timing analysis from a devnet using that tool.

Related items:

• [NV13/Simulation] Measure state-tree growth. #6478
• [NV13/Simulation] Reclaim Space #6476
• Export Metrics #6424

[BigLep]

For 1, @ZenGround0 has a spreadsheet for the estimation.

Scaling:

1. Individual object size. Programatically construct a state tree of some size.

• Devnet too slow
• Manually generate the state tree that is massive
• Potentially let this be a devnet seed? (lotus-shed tool)

Other conversation notes:

• Minimum: add a step to our release process
• Look into testground

[Kubuxu]

We should also evaluate the scaling cost of corn, as well as, state churn (amount of state that gets rewritten per epoch on average) as this determines size of full nodes and snapshots even more so than state size itself.

[Stebalien]

Luckily, in the "good" case, cron only touches partitions if something goes wrong, and never touches sectors. But yeah, we need to consider the "everyone faults all at once" case.

[Stebalien]

So, I think the whole dev-net part of this is a no-go, unfortunately, because we'd actually need to prove the 2k sectors.

Instead, I'm going to go for a local simulation.

[Acumes]

I want to deploy 2K internal network in the test network to verify this function. What should I do?

[Stebalien]

Figuring that out is part of the point of this project. However, actually running this instead of just simulating it may not be possible. Assuming a 10x growth (50EiB target), you'd need:

1. At least 3TiB of storage. Plus overhead so let's call it 10TiB.
2. Enough compute to compute ~500 window posts a minute (or fake it).

So my current plan is to simulate parts of it.

[BigLep]

2021-05-21 discussion:
~2 days for setting up the simulation
~3 days of analysis
EOW 2021-05-24 will have a sense of how we're looking

[Stebalien]

So, we're definitely going to need to do some devnet testing, ideally with fake proofs so we can make realistic seal batches. That won't let us "project" into the future, but will catch bugs like #6338.