Continuous Trie Log Pruning #6075
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Simon Dudley <[email protected]>
Separate out the concepts of world state caching from trie log management Make AbstractTrieLogManager a concrete implemenation (to be further renamed/refactored next commit) Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Feature toggled by --Xtrie-log-pruning-enabled Each time a trie log is persisted, the current trie log is cached and the pruner is run against the oldest entries in the cache. This makes no attempt to manage the backlog of old trie logs, it will only prune what has been added to the cache, i.e. trie logs that have been added since the feature was enabled. Pruner limit exists in case of exceptional circumstances, but we should only ever be pruning all the forks for a single block number during each prune execution. Signed-off-by: Simon Dudley <[email protected]>
Was only used for supporting test code and can instead reuse static factory from InMemoryKeyValueStorageProvider Signed-off-by: Simon Dudley <[email protected]>
|
siladu
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @Override | ||
| public void reset() {} | ||
| public void saveTrieLog( |
Check warning
Code scanning / CodeQL
Non-synchronized override of synchronized method
siladu
added
TeamGroot
Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Inialised once on startup Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
|
Abandoning this approach in favour of an adapted form of the original #6026 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
--Xtrie-log-pruning-enabledTrieLogManager.saveTrieLogThis is a second take of #6026
Currently built on #6072 (diff: siladu/besu@refactor-trie-log-manager...trie-log-pruning-take2)
One downside of this PR compared to #6026 is that restarting besu may leave a gap in the pruned trie logs because we don't preload the cache. This will have to be considered as part of managing the backlog. It may mean that we can't simply rely on a "one-off" resync or prune subcommand. However, these un-pruneable trie logs should be negligible compared to the saving gained for long running nodes.