Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modernized ChainStorage Sync: Enable Chainstorage consuming ChainNode/ChainStorage block as backfill source #66

Open
leozc opened this issue Nov 9, 2023 · 2 comments
Open

Modernized ChainStorage Sync: Enable Chainstorage consuming ChainNode/ChainStorage block as backfill source #66

leozc opened this issue Nov 9, 2023 · 2 comments
Labels
help wanted Extra attention is needed

Comments

@leozc
Copy link
Contributor

leozc commented Nov 9, 2023 •
edited
Loading

  1. ChainNode as data source: Unlike normal node providers, Chainnode uses header to capture security token (yes, safer!) but currently Chainstorage doesn't seem to support this way to authenticate with a block provider.
  2. ChainStorage as data source: Enable ChainStorage use another ChainStorage Blocks to sync can be 100x of the syncing performance (Assume 100 trx for each block)

Cc @jiezhang
#P1
@leozc leozc added the help wanted Extra attention is needed label Nov 9, 2023
@leozc leozc changed the title Enable Chainstorage consume chainnode block as backfill source Enable Chainstorage consuming chainnode block as backfill source Nov 9, 2023
@leozc leozc changed the title Enable Chainstorage consuming chainnode block as backfill source Enable Chainstorage consuming ChainNode/ChainStorage block as backfill source Feb 20, 2024
@leozc leozc changed the title Enable Chainstorage consuming ChainNode/ChainStorage block as backfill source Modernized ChainStorage Sync: Enable Chainstorage consuming ChainNode/ChainStorage block as backfill source Feb 20, 2024
@bestmike007
Copy link
Contributor

bestmike007 commented Feb 20, 2024 •
edited
Loading

Replicator workflow POC: backfilling from another chainstorage: #91

@bestmike007
Copy link
Contributor

Improve replicator workflow by introducing mini batches and continue workflow as new to prevent timeout: #94

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bestmike007 @leozc