FIP-0047: First draft of proof expiration and security policy #446
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,236 @@ | ||
| --- | ||
| fip: 0047 | ||
| title: Proof Expiration & PoRep Security Policy | ||
| author: Jakub Sztandera (@Kubuxu), Irene Giacomelli (@irenegia), Alex North (@anorth), Luca Nizzardo (@lucaniz) | ||
| discussions-to: https://github.com/filecoin-project/FIPs/discussions/415 | ||
| status: Draft | ||
| type: Technical | ||
| category: Core | ||
| created: 2022-08-31 | ||
| --- | ||
|
|
||
| ## Simple Summary | ||
|
|
||
| Adds a `ProofExpiration` parameter to each sector, | ||
| and describes a policy to be adopted in case a flaw is discovered in the theory or implementation of proof-of-replication. | ||
|
|
||
| ## Abstract | ||
| We are not aware of any PoRep issues at this time, but it is crucial to have a plan to deal with any possibility. | ||
| This proposal introduces a concrete mechanism for processing all sectors, | ||
| and describes (but does not implement) a policy that uses the mechanism. | ||
|
|
||
| - The mechanism is to decouple sector `expiration` (renamed to `CommitmentExpiration`) | ||
| and proof expiration (a new sector property is introduced called `ProofExpiration`), | ||
| and introduce a method `RefreshProofExpiration()` that can be used to request a refreshed proof expiration. | ||
| A sector’s commitment may then be longer than its proof expiration. | ||
| - The policy is that, in case of a PoRep flaw, | ||
| the `RefreshProofExpiration` method is disallowed for sectors sealed with the vulnerable code. | ||
| In order to maintain power, a provider must seal a new replacement sector before each existing sector’s `ProofExpriration` epoch is reached. | ||
| If not replaced, a sector with a proof expiration before its commitment expiration will | ||
| incur a termination fee (for example, the current sector early-termination penalty). | ||
|
|
||
| ## Motivation | ||
| The Filecoin network uses cryptographic techniques to provide assurance of the physical uniqueness | ||
| of sectors of data (proof of replication, or PoRep) and their ongoing availability (proof of space-time, PoSt). | ||
| These mechanisms provide the proof of resources underlying the blockchain’s security | ||
| (in addition to the security offered by pledge collateral stake). | ||
| Some of the cryptography involved has been developed relatively recently. | ||
| It is possible that there are errors in either the theory or implementation, | ||
| or that errors may be introduced one day, that undermine the desired assurances. | ||
| The result of such an error would most likely be that storage providers could “cheat” the network | ||
| to claim they were maintaining more committed storage than they in fact possessed. | ||
| This would reduce network security as consensus power could be gained without | ||
| the expected physical infrastructure commitment. | ||
| It is also unfair to non-cheating providers, assuming knowledge of the flaw was limited. | ||
| In the case of an error in PoRep, it is likely that there would be no possible protocol change | ||
| that could detect the cheating sectors after commitment. | ||
|
|
||
| This situation has already arisen once in the life of the Filecoin network. | ||
| The v1.1 PoRep algorithm [patched a bug in the v1 PoRep implementation](https://github.com/filecoin-project/lotus/pull/4873#issuecomment-758099163) | ||
| that weakened the security assurances of sectors. | ||
| The bug was responsibly reported to the Filecoin team and it is unknown if it was ever exploited by a provider. | ||
| We are not aware of any similar bugs at this time. | ||
| Filecoin storage is secure as far as we can ascertain. | ||
|
|
||
| ### Current policy | ||
|
|
||
| The network today has an implicit policy on what to do if another such bug is detected: | ||
|
|
||
| - Implement a new, fixed PoRep algorithm | ||
| - In a network upgrade, the old algorithm is disallowed and the new one mandated for new sectors | ||
| - Old sectors are prohibited extension beyond the 1.5-year maximum commitment | ||
| - The power attributed to old sectors decreases as they expire and is extinct after 18 months. As new sectors are sealed, the insecure *proportion* diminishes even faster. | ||
|
|
||
| The policy might thus be summarised as: | ||
| put up with the potentially-insecure power for a limited period of time, | ||
| but retain existing commitments of providers to the network and vice-versa. | ||
| The 1.5-year window was selected as a compromise: | ||
| a longer maximum commitment would be beneficial for storage stability, | ||
| but a shorter bound improves the response time in case of a bug. | ||
|
|
||
| This policy depends on the 1.5y commitment expiration, | ||
| and would lose effectiveness if the maximum commitment duration were raised. | ||
|
|
||
| ### Changing the maximum sector commitment duration | ||
| The **sector commitment duration** (currently 1.5 years) is the period availability | ||
| a provider commits to when first proving a sector. | ||
| When a sector is close to expiration, a provider can extend the sector for up to the same duration again. | ||
| This can be repeated until the **sector maximum lifetime** (currently 5 years), | ||
| after which further extensions are prohibited. | ||
|
|
||
| It would be desirable to allow storage providers to make longer commitments to sectors (e.g. as proposed in FIP-0036). | ||
| Longer commitments support stability of network’s committed storage, could immediately support longer deals, | ||
| and generally reduce the overheads of cycling sectors into the future. | ||
| A higher possible maximum commitment duration would be beneficial regardless of any incentives toward longer commitments. | ||
|
|
||
| The authors assert that the implementation of a concrete mechanism, and rough consensus around policy, | ||
| should be pre-requisite to increasing the maximum sector commitment duration. | ||
|
|
||
| ## Specification | ||
|
|
||
| ### Mechanism | ||
| - The existing sector `Expiration` property is renamed to `CommitmentExpiration` with | ||
| a duration range from `MinSectorLifetime` to `MaxSectorLifetime`. | ||
| - A new sector property is introduced called `ProofExpiration`. | ||
| The value of `ProofExpiration` is not freely chosen by the SP, | ||
| but takes a value that is derived from and quantised to the sector’s activation epoch. | ||
| - During activation, It is calculated as: `ProofExpiration = SectorActiviation + MaxProofDuration` | ||
| - On refresh: `ProofExpiration = ProofExpiration + (MaxProofDuration - ProofRefreshWindow)` | ||
| - If a sector’s `ProofExpiration` epoch is reached before its `CommitmentExpiration`, | ||
| the sector is terminated and the provider pays a termination fee (the same value as for manual termination). | ||
| - The storage provider can call a new method `RefreshProofExpiration(SectorSelector)` requesting a refreshed proof expiration. | ||
| This call is only available after `ProofExpiration - ProofRefreshWindow` and is disallowed past the `ProofExpiration`. | ||
| It will increase the `ProofExpiration` of each of the requested sectors by `MaxProofDuration - ProofRefreshWindow`. | ||
| - When extending a sector's commitment expiration, the existing `ExtendSectorExpiration` method | ||
| also refreshes the sector's proof expiration, if it is within the proof expiration window. | ||
|
|
||
| This mechanism is implemented by adapting the existing expiration queue mechanism in the miner actor | ||
| to schedule sectors according to the earlier of their proof expiration or commitment expiration. | ||
|
|
||
| #### Parameters | ||
| Two new protocol parameters are defined: | ||
| - `MaxProofDuration`: the maximum period from last commitment or refresh | ||
| for which a PoRep is valid, set to 1.5y to match current behaviour; | ||
| - `ProofRefreshWindow`: the window of time before ProofExpiration epoch | ||
| when the proof can be extended, set to 4 months. | ||
|
|
||
| These values together support a `MaxSectorLifetime` of 5 years with only 3 proof refreshes, | ||
| which is the same number of commitment extensions currently required. | ||
|
|
||
| The existing `MinSectorExpiration` and `MaxSectorExpirationExtension` parameters now refer | ||
| to the commitment expiration. | ||
| A new, larger, value for `MaxSectorExpirationExtension` may be considered. | ||
| - If the current value of 1.5 years is retained, then a sector's initial commitment will always | ||
| expire before its proof expiration is reached. | ||
| - If the value is increased, then a sector may take an initial commitment longer than the proof | ||
| expiration, and so must be refreshed. | ||
|
|
||
| If `MaxSectorExpirationExtension` is increased, then the built-in storage market's | ||
| maximum deal duration is increased to match. | ||
| This could permit deals up to 5 years in duration. | ||
|
|
||
| ### Policy | ||
| If we were to discover a flaw in PoRep theory or implementation: | ||
|
|
||
| - A new, fixed PoRep algorithm would be implemented; | ||
| - In a network upgrade, the old algorithm is disallowed and the new one mandated for new sectors; | ||
| - The `RefreshProofExpiration` method is disallowed for old sectors (ie, created with the vulnerable PoRep algorithm), | ||
| in particular, all old sectors will terminate at `ProofExpiration` epoch; | ||
| - A new method `ReplaceSector(OldSector, NewSector)` can be called by a provider to replace an old sector with a newly-sealed one. | ||
| This requires a Storage Provider to run the sealing procedure with the new PoRep algorithm on the new sector | ||
| (ie, successful call to `PreCommitSector` and `ProveCommitSector`). Note that: | ||
| - The sealing procedure and replacement must be completed before the `ProofExpiration` epoch of the old sector; | ||
| - If the replacement sealing procedure is successful, the old sector is removed with no termination fee; | ||
| - If the replacement sealing is not performed for an old sector before its proof expires, | ||
| that sector is terminated at the `ProofExpiration`. | ||
|
|
||
| This proposal requires implementing the proof expiration mechanism, | ||
| but does not require implementing the sector replacement mechanism until such time as it’s needed. | ||
| Details of that mechanism may depend on the new PoRep algorithm. | ||
|
|
||
| However, acceptance of this proposal should be considered as ratification of this policy in general, | ||
| so that emergency-response implementers can immediately design towards it. | ||
|
|
||
| ### Migration | ||
| During migration at network upgrade, each sector’s ProofExpiration is set to that value it would have | ||
| if it had initially been set to `SectorActiviation + MaxProofDuration` and then refreshed | ||
| until the proof expiration is no less than the commitment expiration. | ||
|
|
||
| ## Design Rationale | ||
| Decoupling of sector commitment duration from proof duration allow providers to make | ||
| long sector commitments (see [FIP0036](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0036.md)) | ||
| without comprising the possibility of executing an “emergency” PoRep algorithm upgrade. | ||
| The mechanism of explicit proof refresh must be implemented prior to recognizing any sectors with longer commitments, | ||
| because it would be generally infeasible to schedule an orderly iteration over all sectors after the fact. | ||
| This design implements a way to cycle over all sectors every 1.5 years. | ||
| Note that, in response to a serious PoRep flaw, | ||
| it would be possible to speed up the cycling by processing the proof expiration queue faster than 1 epoch per epoch. | ||
|
|
||
| A sector’s `ProofExpiration` epoch is strictly derived from its activation epoch | ||
| in order to spread expirations out relatively evenly, | ||
| and with limited opportunity for storage providers to overload the network by scheduling them all at once. | ||
|
|
||
| The cost of this approach is approximately the same as the existing process for extending sector expirations every 1.5 years. | ||
|
|
||
| This proposal also outlines the policy for a disaster-response upgrade in order to | ||
| align the community on a carefully designed plan and establish a foundation for | ||
| a smooth network upgrade in the event of discovery of an insecure PoRep. | ||
| The proposed Policy disallows extending the proof duration for any sectors sealed with vulnerable code | ||
| in order to have the power attributed to insecure sectors decreases gradually as their proof validities expire. | ||
| On the other side, sealing new secure sectors in order to maintain power is allowed by | ||
| the replacement sealing and incentivized by applying a fee for sectors that are not replaced. | ||
|
|
||
| We hope that broad policy agreement now can facilitate smooth governance in response to any future emergency. | ||
|
|
||
| ## Backwards Compatibility | ||
|
|
||
| This upgrade changes the schema for sectors and the miner expiration queue, | ||
| and thus requires a network upgrade and sector state migration. | ||
|
|
||
| Impact on tooling should be minimal beyond changes to sector management software. | ||
|
|
||
| ## Test Cases | ||
| Will be included with implementation to be presented. | ||
|
|
||
| ## Security Considerations | ||
| Decoupling sector proof expiration from their commitment duration allows longer commitment durations | ||
| without compromising the network’s ability to respond to flaws in PoRep. | ||
|
|
||
| If a flaw is discovered, an increased commitment duration would create the need for replacement sealing, | ||
| or penalties for non-replaced sectors. | ||
|
|
||
|
|
||
| TODO: describe the cryptoecon modelling and the simulation outcome | ||
|
|
||
| ## Incentive Considerations | ||
| This proposal does not adjust incentives for providers or clients, | ||
| but it does make longer sector commitments possible. | ||
| Other proposals may then introduce incentives for such commitments. | ||
|
|
||
| In case of an emergency response, | ||
| charging some termination fee when a sector proof is not refreshed creates an incentive | ||
| to seal replacement sectors vs let old sectors expired un-replaced. | ||
| Further analysis is warranted into the _ideal_ termination penalty in this case. | ||
| However, such analysis need not prevent us adopting this mechanism, for its other advantages, | ||
| and policy, in order to provide greater clarity for storage provider business operations. | ||
|
|
||
| ## Product Considerations | ||
| If the maximum sector commitment duration is increased, | ||
| then a sector committed for longer than its initial proof expiration must be refreshed, | ||
| or it will be terminated with a penalty. | ||
| This is a different operational position than a sector committed only as as long as the proof duration, | ||
| and then extended. | ||
| In the latter case, because the sector was not committed to a longer period, | ||
| no termination penalty is applied if the extension is missed. | ||
| Committing a sector for longer than the initial proof duration thereby introduces some | ||
| operational risk to the provider, | ||
| which must remember to and succeed in refreshing the proof within the refresh window. | ||
|
There was a problem hiding this comment. If there is no algo issue, why put the burden on the SP to remember to send this message? There was a problem hiding this comment. It is practically necessary for them to trigger the state accounting. I.e., this isn't proposed because we necessarily think it's a nicer product experience for SPs, but because we can't solve the problem effectively otherwise. |
||
|
|
||
| ## Implementations | ||
| The `CommitmentExpiration` and `ProofExpiration` separation implementation is available at | ||
| https://github.com/filecoin-project/builtin-actors/pull/518. | ||
|
|
||
| The replacement sealing implementation is deferred until needed. | ||
|
|
||
| ## Copyright | ||
| Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if the network upgrade with the new algo occurs 1 day (or some short time period) before the ProofExpiration Date? SP's will not have time to reseal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The SP has 4 months before the network upgrade to trigger the refresh. Which will give them 1.16yr at that point.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't that assume that they will have a lot of notice that this change is coming or that SP's always just refresh at the beginning of the 4-month window? In any case, would be better if there was auto-refresh and the SP only had to worry about this if there were a need to reprove in which case they were given ample time to address.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The community is given at least more than 6-8 weeks of notice for any network upgrade (except for the unexpected urgent ones), and that should be plenty of time for SP to extend the expiration if they want. (Tho I don’t think any good SP would want to refresh bad proofs)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, there is some assumption here of some notice about the change, but I think it's reasonable. It's practically impossible for us to make such a change with no notice, although at our best we have in the past done it with <1week.