Create initial transaction crawler for the mempool #2646
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
teor2345
reviewed
Aug 20, 2021
|
I've updated the code. I don't know if this is useful, but here's a link to what has changed: https://github.com/jvff/zebra/compare/mempool-crawler-r1..mempool-crawler-r1-fixes. |
teor2345
reviewed
Aug 23, 2021
The mempool crawler is responsible for periodically asking peers for transactions to insert into the local mempool. This initial implementation will periodically ask for transactions, but won't do anything with them yet. Also, the crawler is currently configured to be always enabled, but this should be fixed to avoid crawling while Zebra is still syncing the chain.
Prevent the crawler from getting stuck if there's communication with a peer that takes too long to respond.
Spawn a task for the crawler when Zebra starts.
Create a mock for the `PeerSet` service to intercept requests and verify that the transaction requests are sent periodically.
Make it simpler to select the features for test builds. Co-authored-by: teor <[email protected]>
Make it easy to navigate from the `TODO` comment to the current project planning. Co-authored-by: teor <[email protected]>
Make it easy to navigate from the `TODO` comment to the current project planning. Co-authored-by: teor <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
As part of the work to implement a mempool for Zebra, there's a need to crawl peers for transactions to populate the mempool. The mempool crawler is a background task that periodically asks a few peers for transactions.
Specifications
The Bitcoin documentation contains a small description about the mempool related network messages.
Designs
Solution
A
Crawlertype is created inside themempoolcomponent. CallingCrawler::spawnwill construct the type and spawn a task to run it on the background. The crawler periodically requests a few peers (configured by a fan-out constant) for transaction IDs.There's a single test to make sure that the requests are sent in a timely manner to the
PeerSetservice.Review
@teor2345 has been following the design of this, but anyone else can chime in as well.
Reviewer Checklist
Follow Up Work
TransactionIdsresponses to the download and verify stream #2650