feat: message transfer mechanism #2688
Conversation
|
You can find the image built from this PR at Built from 6dedc86 |
|
You can find the image built from this PR at Built from 6dedc86 |
| @@ -243,8 +243,49 @@ proc mountWakuSync*(node: WakuNode): Result[void, string] = | |||
|
|
|||
| return ok((elements, cursor)) | |||
|
|
|||
| #TODO add sync callback and options | |||
| node.wakuSync = WakuSync.new(peerManager = node.peerManager, pruneCB = some(prune)) | |||
| let transfer: TransferCallback = proc( | |||
There was a problem hiding this comment.
I'm not a huge fan of us extending the waku_node module with (more) protocol integration logic. To me it seems like we're mounting a new type of protocol - mountWakuStoreSync() if you will. WakuStoreSync contains a reference to the local wakuStoreClient, wakuArchive and wakuSync and handles the coordination logic between these layers. I imagine this coordination will even be extended in future (e.g. with configuration checks to ensure the archive and sync stores are reasonably dimensioned). WDYT?
There was a problem hiding this comment.
I don't like it either but a "component" (like WakuStoreSync) referencing other components is a BIG no-no in my book and architecturally we don't have hierarchies of components in nwaku and that's a good thing!
I think we have to tolerate a waku_node like this until we have proper channels implemented in chronos then we can move away from callbacks, get rid of waku_node and move to a channel based architecture a la Go or Rust (multi-threading for "free" when chronos impl. it would be another benefit). The fact that waku_node cannot be implemented in Rust should tell you that it's a 🦶 🔫. The only reason waku_node can exist is because of GC magic and multi-threading not being designed for in nwaku.
I believe that whatever functionality we want from WakuStoreSync can be achieved by coordinating components and proper configuration.
|
Basic Store Sync E2E tests done! |
* feat: Waku Sync Protocol * feat: state machine (#2656) * feat: pruning storage mehcanism (#2673) * feat: message transfer mechanism & tests (#2688) * update docker files * added ENR filed for sync & misc. fixes * adding new sync range param & fixes --------- Co-authored-by: Ivan FB <[email protected]> Co-authored-by: Prem Chaitanya Prathi <[email protected]>
Add a callback to query the store of the other node for the missing messages and then add them to our own.