feat: redeclare storage when unsealed state is checked #1377
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is currently needed as lotus doesnt automatically update when the fs changes
*Update default duration to 12hours from 1hour *Allow users to disabl redeclaration of storage chore: make gen
jacobheun
force-pushed
the
feat/storage-redeclare-job
branch
from
e0ae2f2 to
487a90a
Compare
nonsense
approved these changes
Apr 16, 2023
dirkmc
approved these changes
Apr 17, 2023
|
Moved this back to the original hour as default. |
jacobheun
added a commit
that referenced
this pull request
Apr 17, 2023
* feat: monitor sectors unsealed state * feat: announce deals to indexer as newly unsealed or no longer unsealed * feat: test for unsealed state manager * feat: update metadata if seal state changes, call NotifyRemove if sector is removed * feat: better logging * feat: check sector seal state once per hour * feat: act on all deals in a sector * feat: announce sealing state changes for legacy deals also * fix: gfm type * chore: fix merge miss * feat: redeclare storage when unsealed state is checked (#1377) * feat: redeclare storage when unsealed state is checked This is currently needed as lotus doesnt automatically update when the fs changes * chore: fix linting * feat: make unsealedstatemanager configurable *Update default duration to 12hours from 1hour *Allow users to disabl redeclaration of storage chore: make gen * refactor: move storage list to 1hour interval --------- Co-authored-by: Jacob Heun <[email protected]> Co-authored-by: Jacob Heun <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds to #1191 to call storage redeclare on the lotus-miner. Lotus doesn't currently automatically detect fs changes, so this is needed to ensure we're able to automatically keep indexes and their metadata up to date.
Speaking with the Lotus team, this call should be cheap so running it regularly shouldn't be an issue.
Screenshot from devnet
This is a screenshot of the before and after
lotus-miner storage list. On the left the list shows 2 of 4 sectors as being unsealed, but we can see by the screenshot that there is only 1 unsealed sector. After the logs of the job execute (on the right), the storage list then correctly shows 1 of 4 sectors as unsealed.Note on Frequency of this check
It may be useful to allow configuring how often the unseal manager is run. Currently it runs every hour, but in reality it doesn't need to run that frequently. 12 or even 24h is likely a more reasonable default as we ultimately just care about eventual consistency and hour to hour changes aren't likely to be very frequent.
Edit: I set this to 12 hours for now, instead of the current 1 hour.
Remaining Work