Default validator support #525
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
MarcoPolo
approved these changes
Feb 28, 2023
MarcoPolo
approved these changes
Feb 28, 2023
|
Is there a risk here if messages are delivered out of order? Like if a node receives message 5 before message 4 from the same peer, will that node then never forward message 4 on to its other peers? Does that risk some messages not being well propagated to the whole network? |
|
This can only happen if one message from the peer is dropped (because of a full queue) and the next message propagates before the dropped message arrives from other paths. So in theory there is some risk, but this would be quite difficult to trigger. Worth noting, but i am not convinced it is a problem. |
|
There is a simple way to alleviate concerns here: keep a bitvector (say 32 or 64 bits, as an integer) that encodes whether the last few messages have been delivered. How does that sound? |
|
@vyzo I don't think I fully understand how the bitvector would work, but I think keeping a small cache of the last few seqnos and whether they've been delivered once before does make sense and would address the issue I brought up! |
|
Ok, i will implement something along these lines. |
yhassanzadeh13
added a commit
to yhassanzadeh13/go-libp2p-pubsub
that referenced
this pull request
Jul 3, 2023
* feat: expire messages from the cache based on last seen time (libp2p#513) * feat: expire messages from the cache based on last seen time * chore: minor renaming * fix: messages should not be found after expiration * chore: editorial * fix: use new time cache strategy consistently * fix: default to old time cache and add todo for background gc * chore: update to go-libp2p v0.25 (libp2p#517) * Update to go-libp2p v0.25 * Use go 1.19 * chore: update go version and dependencies (libp2p#516) * fix(timecache): remove panic in first seen cache on Add (libp2p#522) * Refactor timecache implementations (libp2p#523) * reimplement timecache for sane and performant behaviour * remove seenMessagesMx, take advantage of new tc api * fix timecache tests * fix typo * store expiry, don't make life difficult * refactor common background sweep procedure for both impls * add godocs to TimeCache * Default validator support (libp2p#525) * add default validator support * add an implementation for basic seqno as nonce validation * missing return * the nonce belongs to the origin peer * add note about rust predicament * add seqno validator tests * minor test tweak, ensure at least 1ms before replay * Fix Memory Leak In New Timecache Implementations (libp2p#528) * fix bug * add for last seen cache * chore: Update .github/workflows/stale.yml [skip ci] * chore: Update .github/workflows/stale.yml [skip ci] * upgrades libp2p version * upgrades libp2p version --------- Co-authored-by: Mohsin Zaidi <[email protected]> Co-authored-by: Marco Munizaga <[email protected]> Co-authored-by: RichΛrd <[email protected]> Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: vyzo <[email protected]> Co-authored-by: Nishant Das <[email protected]> Co-authored-by: GitHub <[email protected]>
yhassanzadeh13
added a commit
to yhassanzadeh13/go-libp2p-pubsub
that referenced
this pull request
Feb 20, 2024
* feat: expire messages from the cache based on last seen time (libp2p#513) * feat: expire messages from the cache based on last seen time * chore: minor renaming * fix: messages should not be found after expiration * chore: editorial * fix: use new time cache strategy consistently * fix: default to old time cache and add todo for background gc * chore: update to go-libp2p v0.25 (libp2p#517) * Update to go-libp2p v0.25 * Use go 1.19 * chore: update go version and dependencies (libp2p#516) * fix(timecache): remove panic in first seen cache on Add (libp2p#522) * Refactor timecache implementations (libp2p#523) * reimplement timecache for sane and performant behaviour * remove seenMessagesMx, take advantage of new tc api * fix timecache tests * fix typo * store expiry, don't make life difficult * refactor common background sweep procedure for both impls * add godocs to TimeCache * Default validator support (libp2p#525) * add default validator support * add an implementation for basic seqno as nonce validation * missing return * the nonce belongs to the origin peer * add note about rust predicament * add seqno validator tests * minor test tweak, ensure at least 1ms before replay * Fix Memory Leak In New Timecache Implementations (libp2p#528) * fix bug * add for last seen cache * chore: Update .github/workflows/stale.yml [skip ci] * chore: Update .github/workflows/stale.yml [skip ci] * bump golang.org/x/net from 0.4.0 to 0.7.0 (libp2p#520) Bumps [golang.org/x/net](https://github.com/golang/net) from 0.4.0 to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](golang/net@v0.4.0...v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: topicscore params can't be set for dynamically subscribed topic (libp2p#540) * fix: topicscore params can't be set for a topic subscribed after gossipsub is initialized * chore:address review comments * Revert "fix: topicscore params can't be set for dynamically subscribed topic (libp2p#540)" (libp2p#541) This reverts commit aa5fd79. * remove usage of deprecated peerid.Pretty method (libp2p#542) * chore: update go-libp2p to v0.32 (libp2p#548) * chore: Update .github/workflows/stale.yml [skip ci] * make tidy --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Mohsin Zaidi <[email protected]> Co-authored-by: Marco Munizaga <[email protected]> Co-authored-by: RichΛrd <[email protected]> Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: vyzo <[email protected]> Co-authored-by: Nishant Das <[email protected]> Co-authored-by: GitHub <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Prem Chaitanya Prathi <[email protected]> Co-authored-by: Sukun <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds support for default validators and provides an implementation that validates the message seqno as a per peer nocne to prevent infinite propagation when the network diameter exceeds the seen cache span.
See #524
Implements the logic described in ipfs/kubo#9665
TBD: