Waku v2: Decide what to do about seqno field

[oskarth]

Problem

Currently the alpha spec says (https://specs.vac.dev/specs/waku/waku-v2.html#protobuf):

The seqno field MAY be used to provide a linearly increasing number. See PubSub spec for more details.

There are some trade-offs involved in using this or not.

Acceptance criteria

1. Decision on what to do with seqno (MAY/SHOULD/SHOULD NOT/MUST/MUST NOT/etc)
2. Rationale in spec

Details

From Pubsub spec:

The seqno field is a 64-bit big-endian uint that is a linearly increasing number that is unique among messages originating from each given peer. No two messages on a pubsub topic from the same peer should have the same seqno value, however messages from different peers may have the same sequence number, so this number alone cannot be used to address messages. Notably the 'timecache' in use by the go implementation contains a message_id, which is constructed from the concatenation of the seqno and the from fields. This message_id is then unique among messages. It was also proposed in #116 to use a message_hash, however, it was noted: "a potential caveat with using hashes instead of seqnos: the peer won't be able to send identical messages (e.g. keepalives) within the timecache interval, as they will get rejected as duplicates."

Possible Solutions

1. Encourage use of seqno for data consistency
2. Discourage use of seqno to not leak that information
3. Something in between (don't lean on it but still support, be agnostic, etc)

Notes

Misc notes from chat log:

• seqno might be used for data consistency
• downside is this message payload is visible to peers
• could there be a modality depending on what metadata protection guarantees we want?
• currently not implemented in nim-libp2p but WIP PR
• useful for finding out what data you are missing, e.g. desktop/historical messages
• depends on how topics are used, can be used for fine tuning but inefficient
• main shortcoming is that you only know you are missing messages if you receive a message where you identify gaps (i.e you know you are missing 2 only if you receive 3, if you never receive 3, you might be missing 2/3)
• additionally unless you know the list of peers publishing on that topic, you might not know of new peers
• ergo topic + timestamp is much more efficient in our scenario most of the cases for data retrieval
• metadata: would reveal how many messages sent on a given app topic

h/t @cammellos for discussion

cc @decanus @arnetheduck @dryajov @kdeme @cammellos

[dryajov]

Imo, the seqno is a bit weird and quite hard to implement properly, it's half way thru a counter and a vector clock and as such doesn't meet the criteria for neither. I'm not sure what is the value add of this field as it stand. For example, the spec says it is linearly increasing and unique for each peer, but there is no way of tracking that across peers; gaps in seqno can be due to many external factors, such as dropped messages due to failing validation, network errors, message reordering, etc... Also, right of the gate, uniqueness and linearity requires implementing persistence at the protocol level, something that isn't a requirement for any other protocol. I would have much ratter this being a loosely specified unique number.

[oskarth]

@dryajov Yeah, inclined to agree. It also doesn't seem like the privacy tradeoffs would be worth it for this protocol.

Leaning towards this option:

Discourage use of seqno to not leak that information

[dryajov]

Yeah, that or a random number. This is probably a good issue to raise in the spec itself because I don't see any way of properly implementing/enforcing this.

[oskarth]

It has been done already as part of Eth2 ethereum/consensus-specs#1981 and #150

As soon as nim-libp2p implements it, we can bump

[oskarth]

Decided already, we will use StrictNoSign policy. See #228