No select() when reading stream data

[davidBar-On]

• Version of iperf3 (or development branch, such as master or
  3.1-STABLE) to which this pull request applies:
  master

• Issues fixed (if any): High UDP packet loss, compare with nuttcp #1707

• Brief description of code changes (suitable for use as a commit message):

Suggested solution to make iperf3 UDP performance comparable to nuttcp (and probably other tools). The change is to not use select() when reading stream data. This by replacing in these case the calls to Nread() to calls to Nread_no_select() (new function).

The assumption is that the select() in Nread() was added to make sure that the main thread will not get stuck (when reading control/JSON data), and that it is not needed for the data streams. E.g., the server will not get stuck even if one of its data streams threads is stuck, as the that thread will be canceled at the end of the test.

Although the reported issue was only about UDP, the change was done also for TCP, SCTP, as it seems reasonable to do. If that may cause issues, the change to these protocols may be removed.

[bmah888]

Thanks for the PR! This change is something I've been thinking about since we went to multi-threading, so I'm glad you were able to make this work with positive results. Looking forward to testing this.

[bmah888]

I did some testing with our dev hosts (two older servers connected by a 100GE path, one router hop within the same rack) and saw much less packet loss and greater "goodput" with single-stream UDP performance. (I briefly tested some other scenarios as well, but this addressed the main reason for this code change.)

I was trying to see if there's an easy way to factor out the common code between Nread() and Nread_no_select() but haven't found one yet. (This is not a blocker for merging this change.)

Hoping to get this merged in the next few days after a little more testing.

[davidBar-On]

... easy way to factor out the common code between Nread() and Nread_no_select() ...

The way I considered was to add a parameter to Nread() for whether to do the select(), but I didn't do that as it would require to change all the calls to Nread().

[bmah888]

Looks good...testing showed a significant improvement on our test hosts (reflected in less packet loss and higher goodput for UDP tests).