protocols/kad: Support client mode

[aschmahmann]

From what I understand from #2025 (reply in thread) the Kademlia implementation does not support client mode. Client mode is quite useful as it allows lower quality (e.g. intermittent availability, high latency, low bandwidth, low CPU/RAM/Storage, etc.) nodes to utilize the DHT for discovery without decreasing the quality of the network.

This feature has been very helpful in go-libp2p and in the various consumers of it such as go-ipfs.

---

To implement client mode all that is really required is that the node does not advertise the DHT protocol (e.g. /ipfs/kad/1.0.0). Additionally, if the node is able to switch between client and server mode at runtime then updates to the supported protocols should be communicated via identify-push/identify-delta.

Once we have the basic client-server mode switching then as a follow up it might be a good idea to, as go-libp2p does, have an "automatic" mode that switches between client and server mode based on educated guessing about what is appropriate for the user (e.g. peers behind relays probably shouldn't be servers, and peers who haven't been online for a while and are stable probably shouldn't either).

[koivunej]

I remember searching for specs/writeup regarding this behaviour at the time of introduction to go-ipfs (0.5 maybe?). Closest I can still find for in libp2p/specs or ipfs/go-ipfs this is libp2p/specs#108 which doesn't contain "client." @aschmahmann could the spec be in some other place? However your description of it does sound simple enough.

[mxinden]

Adding a couple of notes here to help getting started.

For the greater picture I recommend reading the Kademlia paper and the Kademlia specification.

As a first iteration, I would recommend making this a configuration option via KademliaConfig and KademliaHandlerConfig.

When in client mode one can use DeniedUpgrade in order to signal that one does not offer the Kademlia protocol. One would need to return DeniedUpgrade in <KademliaHandlerProto as IntoProtocolsHandler>::inbound_protocol and <KademliaHandler as ProtocolsHandler>::listen_protocol.

A server already detects whether the remote supports the Kademlia protocol via the ProtocolStatus construct in handler.rs. Today we confirm a remote both on successful outbound and inbound Substream negotiation (see inject_fully_negotiated_outbound and inject_fully_negotiated_inbound). To properly support client and server mode, one should only confirm a remote on inject_fully_negotiated_outbound.

[mxinden]

Assigning @whereistejas here. Thanks for offering your help.

---

Edit: I can't assign you directly. Though I guess the comment here is good enough.

[thomaseizinger]

I am now working on this, albeit several layers "down the stack".

In particular, to ship this we need a way for all connections handlers on a connection to communicate. That is somewhat tracked in #2680 (the name is stale as the discussion evolved). There is a draft in #3651.

A dependency of that PR is #2831 which I am working on in #3745.

[mxinden]

Discoverability of addresses of clients in client-mode

Roles

• Server node S
• Client node C
• Other node O

Problem

With Kademlia client-mode C will no longer be added to S's routing table. In the case where O asks S of the addresses of C, via the FIND_NODE RPC, S will not be able to tell O the addresses of C as C is not stored in S's routing table.

Possible Solutions

1. Have S keep a cache of addresses of clients like we do in libp2p-identify.

   rust-libp2p/protocols/identify/src/behaviour.rs

   Lines 59 to 60 in df19abb

   	/// The addresses of all peers that we have discovered.
   	discovered_peers: PeerCache,

2. Add some mechanism where the libp2p-kad NetworkBehaviour can ask the other NetworkBehaviour implementations for addresses of C, similar to the NetworkBehaviour::addresses_of_peer method.

[thomaseizinger]

Discussed out-of-band with @mxinden : This can be shipped as an additional PR on top of client-mode in a backwards-compatible way, thus deferring for the initial implementation!