feat(core-snapshots): implement core-snapshots package #3672
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… package, add tests
…store method in core-snapshot package
…pCompression to meta data
# Conflicts: # packages/core-database/src/repositories/index.ts # packages/core-kernel/src/contracts/index.ts
ghost
mentioned this pull request
sebastijankuzner
force-pushed
the
feature/core-snapshots
branch
from
bfbffde to
e181fff
Compare
# Conflicts: # yarn.lock
|
This pull request introduces 2 alerts when merging 6a28684 into 932cfec - view on LGTM.com new alerts:
|
# Conflicts: # packages/core-database/src/index.ts # packages/core-kernel/src/contracts/index.ts # packages/core-kernel/src/exceptions/index.ts
|
This pull request introduces 1 alert when merging f654e30 into 027c6de - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 894a633 into 027c6de - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 9332474 into 027c6de - view on LGTM.com new alerts:
|
|
This pull request introduces 1 alert when merging 1b68e2d into 027c6de - view on LGTM.com new alerts:
|
faustbrian
reviewed
May 4, 2020
__tests__/unit/core-snapshots/filesystem/stream-reader-writer.test.ts
Outdated
Show resolved
Hide resolved
__tests__/unit/core-snapshots/filesystem/stream-reader-writer.test.ts
Outdated
Show resolved
Hide resolved
faustbrian
reviewed
May 4, 2020
__tests__/unit/core-snapshots/filesystem/stream-reader-writer.test.ts
Outdated
Show resolved
Hide resolved
faustbrian
reviewed
May 4, 2020
__tests__/unit/core-snapshots/filesystem/stream-reader-writer.test.ts
Outdated
Show resolved
Hide resolved
faustbrian
reviewed
May 4, 2020
__tests__/unit/core-snapshots/filesystem/stream-reader-writer.test.ts
Outdated
Show resolved
Hide resolved
faustbrian
reviewed
May 4, 2020
faustbrian
reviewed
May 4, 2020
faustbrian
reviewed
May 4, 2020
ghost
deleted the
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds new core-snapshots package with included tests. Changes also exists inside core package for running snapshot package from CLI. Package is rewritten from start and includes similar functionality like previous implementation from 2.6 version. Package use node workers to speed up execution and impact dump method the most. Implemented methods:
Truncate
Remove all blocks, rounds and transactions from database.
Rollback
Rollback database to specific block height or rollback database for specified number of blocks.
Dump
This method will make a snapshot from database and save exported data into the default snapshot location. There is possibility to set range of exported data based on block height parameter, use zlib compression and set desired codec. Meta data are also exported and contains information like exported range, used codec and compression.
Verify
Verify data exported with dump command. Validation is successful when all the blocks, rounds and transactions are successful, otherwise it will log first invalid item and stop execution.
Restore
Will restore all the data from dumped snapshot. Optionally it can be run with truncate method, if this is defined in parameters.
Checklist