Connectionless tests for UDP sockets

[tinybike]

I'm thinking of filling in this repo a bit. Wanted to make sure no one else is working on this already and that I implement this in a way that is useful. Basic question: what is UDP used for in IPFS? (In particular, is it used "as-is", or is there expected to be some application code providing reliability, ordering, etc.?)

I had a look through the abstract-transport tests. The tests are currently set up for connection-oriented (TCP) sockets. Would I just need to tweak the tests a bit to add connectionless (UDP) tests?

[daviddias]

IPFS is designed to be multitransport, that being said, it should work over any transport protocol or addressing system. The way that we do this is by abstracting all of the network nuances with libp2p and using a PubKey+multiaddr addressing scheme to enable a given PeerID to be available through a several transports.

udp is one more protocol that we want to be able to support. As far as libp2p is concerned, a given transport should implement abstract-transport interface https://github.com/diasdavid/abstract-transport#api, that is, whatever happens underneath, the expectations for those API calls should be fulfilled. This doesn't mean however, that we need to add reliability to UDP in order to make it happen (a simple initial signalling message should be enough to verify that a socket was indeed open).

We created the notion that a transport should return a connection and a connection should be upgradable (to a cyphered channel, to a reliable channel, to a multiplex channel, etc) more context here: https://github.com/diasdavid/node-libp2p-swarm/issues/8

libp2p spec - ipfs/specs#19 (easier to read: https://github.com/diasdavid/specs/tree/libp2p-spec/protocol/network)

All this being said (and my apologies if I'm being redundant to the information you already had :)), it would be great to have your help implementing the transport. I'm interested to learn what kind of tests do you have in mind for non reliable connections? Our goal was to have a incremental battery of tests that validates that a transport can indeed fulfil the expectations of the API.

[ghost]

I have a pretty good idea how message-oriented interfaces all the way through libp2p could look like and wanna write these down properly in the next few weeks. @tinybike Please keep me in the loop if you want input or review, or keep nagging me into writing my thoughts down :)

[theobat]

@diasdavid

a way to make it work is to upgrade the UDP socket with some kind of Congestion Window to add reliability to the flow, like UDT or uTP do, however, we don't have that 'Congestion Window' stand alone module implemented.

The problems I understand for udp (but also the inherent unreliable aspects of udp) as you explained in the other issue:

• Packets being unordered.
• Packet losses because congestion or any other under layer reasons (such as radio interferences over wifi)

Now, I understand why this handles the congestion I don't see why it solves the order issue (and the other potential losses), does it ?
And I am not sure I get the upgrades architecture right here, how would the TCP connection look like before it is upgraded with the congestion flow. I mean, it's already there as soon as you get the connection once the socket is open no (because it's tcp)?
I feel like the implicit purpose here is to flatten the transport layer protocols differences with more or less application layer counterparts depending on the protocol, to abstract any network interaction into a single ipfs interface. Is that a good summary ?

But then, how do we ensure that adding the reliability to udp doesn"t break the very purpose of udp ? (such as very low latency ?)

@whyrusleeping mentionned :

we currently use a stream cipher that requires reliable ordered packets.

I have not been able to identify where this is implied in the code go-libp2p-crypto or js-libp2P-crypto
Can you point me to the right direction in there ? (I suppose it's just the fact that the unordered message could not work out for guaranteeing integrity because changing the order would change the hash but I don't see where you are receiving those raw data from the transport if that makes any sense)
And by the way I can't fin any interface for the crypto implementations, seems an issue is already pointing that out but it does not seem "complete". It would be very helpful to have one.

Thanks for your help ! just trying to figure out things :)