User Markets: sector event subscribers and private sector info

[ZenGround0]

This is building off of discussion in #265 FYI @anorth

Event Subscribers in SectorInfo

The above proposal provides a way to do sector event subscription without adding market data to sector infos. The tradeoff is that the registered subscriber will receive all event notifications instead of only those pertaining to the sectors it cares about.

This is unfortunate because in all variants of #298 sectors must inform the miner actor at time of commitment which markets they are interested in, so this data is theoretically available. What we'd really like to see is something close to what the current market actor does. Every sector event is sent over to the market(s) and only the markets that were registered with the sector upon commitment. Can we do better?

One trivial way to do this (forgetting about fault sector info loading limits for now) is to include subscriber addresses in the sector info. During sector lifecycle events we load the sector info, go through all the subscribers and then pass the event to them. However we don't want to put any deal info in sector infos so this is not a direction we've seriously considered so far.

SectorInfo constrained by Data Availability at scale

As I understand it the main objective in removing market specific data from the sector info is to not create expectations that data in the sector info will be available to public callers. This is essential for any scaling solutions that levarage removing public data availability of miner actor state like neutron or zk rollup miner actors. Since we want to be able to scale our systems we can't allow sector users to depend on properties that scalable systems can't provide. Rephrased: we can only allow access to data on chain in a way that solves the data availability problem for future versions of filecoin state that scale.

New idea: have the miner operator solve the data availability problem by default when responding to subscribers. To achieve this all state transitions that can trigger events calling subscribers will either need to provide the data for this part of filecoin state or fail. This is trivial for events like expiration extension and data change. The harder part is cron.

However safe cron by allowing cron failures and recoveries should solve this. In the world where safe cron leads to all handle deadline messages being sent explicitly by SPs the data availability solution is trivial - just include in the message from storage provider. In the world where safe cron still runs handle deadline messages in cron this is more difficult. Naively we could just have all faults fail cron and get recovered manually. We could additionally provide a data availability staging area in miner state. If an SP sees that their operation has faulted and needs to provide data about subscribers they can send a message providing data to the miner actor, and then cron can use this data at the later cron epoch.

Note that this allows state to scale but event processing by subscribers is still a computational bottleneck. Using the safe cron ideas we can restrict this to acceptable levels based on network carrying capacity but this may be too restrictive for even basic subscription callback notifcations at very large network sizes.

Private Sector Info

A way to force ourselves to operate more in this model is to push more of sector info underneath a private hash. Concretely for this proposal we could push the addresses of subscribers to a sector into the leaves of a new private field in sector info which would be an unlinked cid. This way it would be impossible to write code that doesn't solve the data availability problem. We could put all sorts of other things in here included CommD. CommR should even be feasible to move here but would require new PoSt circuits which prove against the private sector info hash instead of CommR directly.

This instantiation also makes the tradeoff clear: users will have more expensive messages including both the data they are making available and merkle proofs of data inclusion for every operation

SectorInfo loading caveat for fault events

If we did per sector info tracking of event subscribers and propagated info about faults we'd immediately run into the problem motivating the design of partitions. We would need to solve fault declarations without loading every sector info. A likely candidate is to take the union of all sector notifications as partition notifications and adding private fields to partitions. This complicates sector assignment as we would need to do data availability on the partition's private subscriber tracking during prove commit in order to complete assignment. However there's no fundamental issue added there that doesn't already exist with partition scaling.

[anorth]

Private sector info is a promising angle for this. If we move state behind a commitment in this way, the data will move out of chain state, but the computation will still happen on-chain any time it is used or modified.

A step further (out of scope here, but maybe later) is off-chain state and computation, which requires a SNARK or similar. But it's interesting to see if there are some parts of state for which keeping the computation on-chain is still ok.

On challenge to this right now is that our gas pricing is wrong, and on-chain storage is not priced above transaction bytes. It should be different by some multiple.