Skip to content
This repository has been archived by the owner on Jun 14, 2024. It is now read-only.

docs: improved filter specification #562

Merged
merged 12 commits into from
May 2, 2023
Merged

docs: improved filter specification #562

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
9dca1ca
docs: improved filter specification
jm-clius Dec 9, 2022
d2d80b6
docs: added pubsub topic to push
jm-clius Dec 9, 2022
4f43a25
fix: formatting fix
jm-clius Dec 12, 2022
29ae394
docs: remove confusing recommendation
jm-clius Dec 13, 2022
cd3e880
docs: specify empty filter criteria
jm-clius Jan 26, 2023
7c47eb0
docs: specify push failure
jm-clius Jan 26, 2023
12793c5
docs: clarify mandatory fields for valid filter criteria
jm-clius Mar 20, 2023
0f02d70
docs: clarify filter client role in verifying MessagePush
jm-clius Apr 20, 2023
7f5cc79
docs: add link to previous filter draft version
jm-clius Apr 26, 2023
d24b589
docs: try to repair yaml block
jm-clius Apr 26, 2023
2c74eed
docs: another attempt to fix yaml block
jm-clius Apr 26, 2023
cb35b95
docs: new suggestion on how to use previous versions
jm-clius May 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
201 changes: 137 additions & 64 deletions content/docs/rfcs/12/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ contributors:

# Content filtering

**Protocol identifier***: `/vac/waku/filter/2.0.0-beta1`
**Protocol identifiers***:
_filter-subscribe_: `/vac/waku/filter-subscribe/2.0.0-beta1`
_filter-push_: `/vac/waku/filter-push/2.0.0-beta1`

Content filtering is a way to do [message-based
filtering](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern#Message_filtering).
Expand Down Expand Up @@ -61,77 +63,148 @@ The following are not considered as part of the adversarial model:
## Protobuf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer the term "Wire format". It is more generic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree! Note that this PR specifically avoids format/heading/layout changes so that only the suggested wire format change can be reviewed. Once done, next step would be to adhere to the template and add theory, implementation, etc. sections.


```protobuf
message FilterRequest {
bool subscribe = 1;
string topic = 2;
repeated ContentFilter contentFilters = 3;

message ContentFilter {
string contentTopic = 1;
syntax = "proto3";

// 12/WAKU2-FILTER rfc: https://rfc.vac.dev/spec/12/
package waku.filter.v2;

// Protocol identifier: /vac/waku/filter-subscribe/2.0.0-beta1
message FilterSubscribeRequest {
enum FilterSubscribeType {
SUBSCRIBER_PING = 0;
SUBSCRIBE = 1;
UNSUBSCRIBE = 2;
UNSUBSCRIBE_ALL = 3;
}

string request_id = 1;
FilterSubscribeType filter_subscribe_type = 2;

// Filter criteria
optional string pubsub_topic = 10;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it mean we can only have subscriptions for one pubsub topic for a given remote node?
I'd assume with topic sharding, a light client may need to subscribe to several pubsub topics (community topic + 1:1 topic at least) and if I understand correctly, any subscription request replaces the previous one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Answered elsewhere, but adding here for completeness:
No, a client can subscribe to content topics within as many pubsub topics as it likes. The limitation here is only that each request can only subscribe to content topics belonging to the same, single pubsub topic. A client can simply send a subsequent request with content topics belonging to another pubsub topic if it's interested in those.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this NOT be optional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is only optional within the context of protobuf field design: some types of FilterSubscribeRequests (e.g. SUBSCRIBER_PING, UNSUBSCRIBE_ALL) SHOULD NOT have a populated pubsub_topic. Within the context of the protocol itself this field is conditionally mandatory (which would require optional in the protobuf), e.g. the pubsub_topic MUST be populated, if the filter_subscribe_type is SUBSCRIBE or UNSUBSCRIBE.

repeated string content_topics = 11;
Copy link
Contributor

@LNSD LNSD Jan 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should write a recommended maximum number of content topics per subscription request.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree and good point! Will add that to the implementation suggestions in a follow-up PR.

}

message MessagePush {
repeated WakuMessage messages = 1;
message FilterSubscribeResponse {
string request_id = 1;
uint32 status_code = 10;
optional string status_desc = 11;
}

message FilterRPC {
string requestId = 1;
FilterRequest request = 2;
MessagePush push = 3;
// Protocol identifier: /vac/waku/filter-push/2.0.0-beta1
message MessagePush {
WakuMessage waku_message = 1;
}
```

#### FilterRPC

A node MUST send all Filter messages (`FilterRequest`, `MessagePush`) wrapped inside a
`FilterRPC` this allows the node handler to determine how to handle a message as the Waku
Filter protocol is not a request response based protocol but instead a push based system.

The `requestId` MUST be a uniquely generated string. When a `MessagePush` is sent
the `requestId` MUST match the `requestId` of the subscribing `FilterRequest` whose filters
matched the message causing it to be pushed.

#### FilterRequest

A `FilterRequest` contains an optional topic, zero or more content filters and
a boolean signifying whether to subscribe or unsubscribe to the given filters.
True signifies 'subscribe' and false signifies 'unsubscribe'.

A node that sends the RPC with a filter request and `subscribe` set to 'true'
requests that the filter node SHOULD notify the light requesting node of messages
matching this filter.

A node that sends the RPC with a filter request and `subscribe` set to 'false'
requests that the filter node SHOULD stop notifying the light requesting node
of messages matching this filter if it is currently doing so.

The filter matches when content filter and, optionally, a topic is matched.
Content filter is matched when a `WakuMessage` `contentTopic` field is the same.

A filter node SHOULD honor this request, though it MAY choose not to do so. If
it chooses not to do so it MAY tell the light why. The mechanism for doing this
is currently not specified. For notifying the light node a filter node sends a
MessagePush message.

Since such a filter node is doing extra work for a light node, it MAY also
account for usage and be selective in how much service it provides. This
mechanism is currently planned but underspecified.

#### MessagePush

A filter node that has received a filter request SHOULD push all messages that
match this filter to a light node. These [`WakuMessage`'s](./waku-message.md) are likely to come from the
`relay` protocol and be kept at the Node, but there MAY be other sources or
protocols where this comes from. This is up to the consumer of the protocol.

A filter node MUST NOT send a push message for messages that have not been
requested via a FilterRequest.

If a specific light node isn't connected to a filter node for some specific
period of time (e.g. a TTL), then the filter node MAY choose to not push these
messages to the node. This period is up to the consumer of the protocol and node
implementation, though a reasonable default is one minute.
## Filter-Subscribe

A filter service node MUST support the _filter-subscribe_ protocol
to allow filter clients to subscribe, modify, refresh and unsubscribe a desired set of filter criteria.
The combination of different filter criteria for a specific filter client node is termed a "subscription".
A filter client is interested in receiving messages matching the filter criteria in its registered subscriptions.
It is RECOMMENDED that filter service nodes allow only one subscription per client.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this recommendation may lead to issue:

  1. Alice subscribes for /content/topic/1 to Bob
  2. Alice does a handshake with Carol, and hence needs to subscribe to /content/topic/2 to Bob

If only one subscription is to be maintain, what should Alice do?

A. unsubscribe and re-subscribe for both content topic?
B. update the subscription to add the new content topic?

A may lead to missed messages
B does not seem possible from the current spec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see the confusion that the current phrasing may lead to. B is indeed possible and the recommended way of doing it.

Note that:

The combination of different filter criteria for a specific filter client node is termed a "subscription".

And this can be modified as you go along by subscribing and unsubscribing from content topics.

The idea is that a service node should not maintain multiple sets of such filter criteria for the same client. This used to be the case with the previous spec, where the same client could register the same filter criteria several times at the same service node, each time using a different request ID. I think I should remove this confusing wording.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this line in 29ae394 as it didn't add to the functionality and would just confuse readers. The point should be that any client can modify their filter criteria as many times as they want, adding and removing content topics on the fly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry my initial comment was not valid, I intended to delete it as I read the rest of the specs. I thin it's clear that Alice should update her subscription as I finished to read the spec.


Since a filter service node is consuming resources to provide this service,
it MAY account for usage and adapt its service provision to certain clients.
An incentive mechanism is currently planned but underspecified.

### Filter Subscribe Request

A client node MUST send all filter requests in a `FilterSubscribeRequest` message.
This request MUST contain a `request_id`.
The `request_id` MUST be a uniquely generated string.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if an user sends two FilterSubscribeRequest with the same request_id? Would that generate an error? or would the latest request received overwrite the subscription parameters of the first one?

Copy link
Contributor Author

@jm-clius jm-clius Dec 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest request received would overwrite the subscription parameters of the first be considered in addition to the previous one - if it SUBSCRIBEs to a different set of content topics as the first, the client will now be subscribed to the content topics of both requests. We suggest now that service nodes only keep one subscription per peer (i.e. per peerId) and simply service requests as they come in (without keeping any requestId state - just making sure responses use the same requestId).
Reusing a request_id is now a spec violation on the side of the client, so the client would have to deal with the ambiguity of two responses on the same request_id (what if one fail and one succeeds?). It could then use the refresh workflow to ensure the desired filter is installed (by just repeating its SubscribeRequest).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, qualifying here: the latest request received would be considered in addition to the parameters of the first. It's possible to build a subscription by sending multiple FilterSubscribeRequests variously adding and removing content_topics. This is an NB change - namely, SUBSCRIBE no longer replaces the entire subscription, it simply subscribes a client to the included filter criteria in addition to what it's already subscribed to.

Copy link
Contributor

@kaiserd kaiserd Dec 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be undefined behaviour as the client would not follow the spec in this case. It is a MUST for the request_id to be unique.
Imo, it would be most consistent if the implementation simply dropped the second request with the same id.

The IETF says: "Be strict in what you send but tolerant in what you accept" (I'm paraphrazing, the original was better 😅 ), but here, accepting the second filter would contradict the spec.

We could add a line making this explicit:

When receiving a second request with the same request_id, the request is silently dropped.

(oh, I wrote this in a not up-to-date browser window where your comments were not there yet @jm-clius 😅 )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imo, it would be most consistent if the implementation simply dropped the second request with the same id.

True, but we've now removed requestId as some identifier for the subscription and we don't (want to) keep track of requestIds in the service node. SUBSCRIBE now simply means subscribing to the included filter criteria (not replacing/removing any existing subscription - just extending it). The requestId, from the service node perspective, now simply ties the response to the request and is discarded directly afterwards by the service node. If it receives a subsequent SUBSCRIBE with the same requestId it will simply try to honor it with no reference to previous requestIds.

Each request MUST include a `filter_subscribe_type`, indicating the type of request.

### Filter Subscribe Response

In return to any `FilterSubscribeRequest`,
a filter service node SHOULD respond with a `FilterSubscribeResponse` with a `requestId` matching that of the request.
This response MUST contain a `status_code` indicating if the request was successful or not.
Successful status codes are in the `2xx` range.
Client nodes SHOULD consider all other status codes as error codes and assume that the requested operation had failed.
In addition, the filter service node MAY choose to provide a more detailed status description in the `status_desc` field.

### Filter matching

In the description of each request type below,
the term "filter criteria" refers to the combination of `pubsub_topic` and a set of `content_topics`.
The request MAY include filter criteria, conditional to the selected `filter_subscribe_type`.
The `pubsub_topic` is always optional in filter criteria.
A `WakuMessage` matches filter criteria when its `content_topic` is in the `content_topics` set
and, if included in the criteria, it was published on a matching `pubsub_topic`.
If the `pubsub_topic` is set and not empty,
a `WakuMessage` will only match the filter criteria if it has a matching `content_topic` and was published on _the same_ `pubsub_topic`.
If the `pubsub_topic` is either empty or unset,
a `WakuMessage` will match the filter criteria if it has a matching `content_topic` and was published on _any_ `pubsub_topic`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we should be more restrictive in the spec and make the pubsub_topic mandatory. So, there should be always a pubsub_topic and a list of content_topic.

  // Filter criteria
  string pubsub_topic = 10;
  repeated string content_topics = 11;

In a scenario where we apply sharding to the Waku relay networks, a node will be subscribed to "several" pub-sub topics. If the gateway node subscribes to a network with the same content topic as our network of interest, the user may receive mixed or duplicated messages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree with you.

This spec was written when there was still ambiguity on whether a content topic strictly match to only one pubsub topic or could span several shards (in which case the mixed/duplicate messages would be desired).

I think we're luckily now moving in a direction where we can see a content topic as existing only within the context of a specific pubsub topic (and in future may even abstract the underlying pubsub topic away). In other words, we can see the (pubsubTopic, contentTopic) vector as the minimal unique filter criterion. Pinging @kaiserd to double check direction here and will then update spec accordingly (needs changes in a couple of places).


### Filter Subscribe Types

The following filter subscribe types are defined:

#### SUBSCRIBER_PING
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still need to finish seeing the point of this message 🤔

In my head: If a client sends a subscribe message, with the server's response, it will be sure that there is a subscription. And additionally, we are already sending a "subscription refresh" request from the client.

On the other hand, I see the value in updating and pruning the subscriptions if the subscriber does not respond to a server-sent ping. But this will overlap with the peer management and the connection's keep-alive mechanisms (e.g., libp2p's ping).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was also unsure how useful this would be, so wanted to get the most common clients' perspective to understand if this has any value. (See e.g. point 4 in the description)

it will be sure that there is a subscription

But the server provides no guarantees to maintain this subscription over time. The client's responsibility is to have some redundancy and a ping mechanism to ensure that its subscriptions remain active.

we are already sending a "subscription refresh" request from the client

Indeed why I wasn't sure if the lightweight ping is necessary, but note that refreshing a complex subscription with many content topics will be expensive, whereas this envisions a much more frequent ping mechanism. It could be a special case of subscription refresh with empty filter criteria, but IMO that would be abusing the mechanism and I'd rather have something explicit.

this will overlap with the peer management and the connection's keep-alive mechanisms

This functions on a different level - it's not about whether there is an active connection. For filter we assume that connections may drop from time to time. For whichever reason, the service node may remove your subscription (e.g. due to a restart, reaching capacity, etc.) and the client needs a lightweight way to ping its subscription - not strictly to keep it alive, but to maintain a minimum set of subscriptions across multiple service nodes.


A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBER_PING`
requests that the service node SHOULD indicate if it has any active subscriptions for this client.
The filter client SHOULD exclude any filter criteria from the request.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if a client sends the ping together with a subscription list? Should the node respond with a failure? Should it just ignore it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Have specified in cd3e880.

The filter service node SHOULD respond with a success code if it has any active subscriptions for this client
or an error code if not.

#### SUBSCRIBE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if a node is not subscribed to a network (a.k.a. pubsub_topic)? Does the node receive an error? Which error code?

I think we should specify the "common" errors. I do not know where, but it is necessary so all the implementations use the same error codes. Something like IETF's RFC 9110: https://www.rfc-editor.org/rfc/rfc9110.html#name-status-codes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree re specifying the common errors. I think this should likely be in a separate BCP RFC with a separate lifecycle (can be draft and extended while this one is stable. This RFC aims to be as minimal as possible in order to define the smallest set of rules that allow for stable operation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some failure scenarios that I find problematic:

Given a client using the Waku filter protocol to subscribe to a Network (pubsub_topic) and a messages stream (content_topic) via a 3rd party gateway node. The 3rd party gateway node may unregister himself from the Network while the subscription is still active (not removed nor expired).

Given a client using the Waku filter protocol to subscribe to a Network (pubsub_topic) and a messages stream (content_topic) via a 3rd party gateway node. The 3rd party gateway node may crash and reboot, losing all subscriptions since they are kept and managed in memory.

In my opinion, these cases will influence the "subscription expiration timeout" in the server and the "subscription refresh cadence" in the client.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed! Both valid scenarios which will happen.

Within the filter protocol there is no mandated guarantee for filter service nodes to maintain, recreate or even just honour existing subscriptions from subscribers. Of course good filter service nodes will be implemented to at least honour existing subscriptions for as long as they can. This is why in the implementation section (follow-up PR) I will describe suggested methods for the client to ensure it maintains reliable filter subscriptions and most tools in this version of the protocol is designed to allow the client to do just that.
e.g.

  • it should have an n degree of redundancy (tradeoff bandwidth vs reliability)
  • it should ping its subscriptions periodically (as service nodes won't persist its subscription or guarantee its longevity)
  • it could refresh its subscriptions periodically, at lower frequency

This of course still does not lead to a "fault-tolerant" or "reliable" filter protocol, which IMO should be a next step and could combine concepts from store and acknowledgements of message UID etc.


A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `SUBSCRIBE`
requests that the service node SHOULD push messages matching this filter to the client.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a subscriber only specifies the pubsub_topic and no content_topic, would they receive all messages that match the pubsub topic regardless of the content type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment on the UNSUBSCRIBE above. In short, pubsub topic is only seen as a qualifier for the content topic. In other words, having no content topics SHOULD be treated as a an empty SUBSCRIBE and SHOULD be ignored. This indeed needs to be clarified.

The filter client MUST include the desired filter criteria in the request.
A client MAY use this request type to _modify_ an existing subscription
by providing _additional_ filter criteria in a new request.
Comment on lines +162 to +163
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is a bit nitpicking, but:

Suggested change
A client MAY use this request type to _modify_ an existing subscription
by providing _additional_ filter criteria in a new request.

I would model every "pubsub_topic+content_topic" as an individual subscription. So a node, identified by its PeerId/NodeId, can have multiple transactions. You either add a new subscription or remove an existing subscription. There is no updating of the subscription.

A subscription request with multiple content_topics is a convenient way of batching multiple subscriptions in a single request without sending multiple requests, one per content_topic.

IMO This simplifies the implementation a lot.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! This is the plan in terms of how it will be implemented.

After trying to settle on terminology I came up with the earlier stated:

The combination of different filter criteria for a specific filter client node is termed a "subscription".

Each filter criterion/transaction will indeed be modelled as a "pubsub_topic+content_topic" which can only be added or removed. I found that people most often spoke about that action as "updating my subscription", but am not set on the terminology.

A client MAY use this request type to _refresh_ an existing subscription
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that subscriptions have an expiration time in the server that should remove a subscription if the node has not refreshed it via SUBSCRIBE within the time.

Is that correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, within the filter service node the protocol does not mandate any management of subscriptions. It would be up to the client to ensure it maintains (redundant) subscriptions across filter service nodes. The implementation section (next) will recommend a maximum interval between ping for clients. Filter service nodes are specified so that they in general "keep as many subscriptions as they can, for as they long as their resources allow". The point being that a client should not expect its subscriptions to be reliable (we want to build a network where it is, of course, this is about specification).

by providing _the same_ filter criteria in a new request.
The filter service node SHOULD respond with a success code if it successfully honored this request
or an error code if not.

#### UNSUBSCRIBE

A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE`
requests that the service node SHOULD _stop_ pushing messages matching this filter to the client.
The filter client MUST include the filter criteria it desires to unsubscribe from in the request.
A client MAY use this request type to _modify_ an existing subscription
by providing _a subset of_ the original filter criteria to unsubscribe from in a new request.
The filter service node SHOULD respond with a success code if it successfully honored this request
or an error code if not.
Comment on lines +172 to +180
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if I a client sends an UNSUBSCRIBE request containing a pubsub_topic and no content_topics? (since a repeated field can contain 0 items)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks for picking this up! It should definitely be clarified in the specs. My view is that SUBSCRIBE and UNSUBSCRIBE (in fact the idea of subscriptions) should be related to content topics, qualified by a (perhaps now mandatory) pubsub topic. That means that only content topics will be considered in subscribe and unsubscribe requests. Not having any content topics will be seen as an empty unsubscribe request and SHOULD be ignored.

Of course, LMK if you believe from client perspective if a different semantic would work better, but the main idea of Filter is to operate on content topics.


#### UNSUBSCRIBE_ALL

A filter client that sends a `FilterSubscribeRequest` with `filter_subscribe_type` set to `UNSUBSCRIBE_ALL`
requests that the service node SHOULD _stop_ pushing messages matching _any_ filter to the client.
The filter client SHOULD exclude any filter criteria from the request.
The filter service node SHOULD remove any existing subscriptions for this client.
It SHOULD respond with a success code if it successfully honored this request
or an error code if not.

## Filter-Push

A filter client node MUST support the _filter-push_ protocol
to allow filter service nodes to push messages matching registered subscriptions to this client.

A filter service node SHOULD push all messages
matching the filter criteria in a registered subscription
to the subscribed filter client.
Comment on lines +199 to +201
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should a client worry about service nodes that push messages when not subscribed to?
This could be the case if for example a content or pubsub topic is RLN protected because processing of messages is costly.
To DDOS a client, a node could unsolicited messages, even tho the client never subscribed to this service node.

The only way a client can protect itself would be to track the peer id of nodes it subscribed too, and ensure push messages are coming from those nodes.
This may also be needed done the line for some scoring, to keep track of the reliability of a node.

I think we should mention this possible attack vector in the spec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed! Very good point. Thanks. In the next PR I want to adhere to the RFC template, which includes a section on security, attack vectors, etc.

These [`WakuMessage`s](./waku-message.md) are likely to come from [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/),
but there MAY be other sources or protocols where this comes from.
This is up to the consumer of the protocol.

If a specific filter client node is not reachable from the service node for a period of time,
the filter service node MAY choose to stop pushing messages to the client and remove its subscription.
This period is up to the service node implementation.
We consider `1 minute` to be a reasonable default.

### Message Push

Each message MUST be pushed in a `MessagePush` message.
Each `MessagePush` MUST contain one (and only one) `waku_message`.
A filter client SHOULD NOT respond to a `MessagePush`.
Since the filter protocol does not include caching or fault-tolerance,
this is a best effort push service with no bundling or retransmission of messages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How should an implementation manage delivery failures? Should it drop the subscription? Should it retry to push every message?

I think that defining a push failure strategy is a complex scenario that the spec cannot leave open to the implementations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I've added this in 7c47eb0.
The idea is to specify this is in such a way that it does not preclude service nodes from attempting retransmissions, but does not build a dependency on this mechanism into the minimal protocol definition either.
In addition we should provide a BCP where we're opinionated on implementation. Good catch!


---
# Future Work
Expand Down