Updated Http4sWebSockets.

[kamilkloch]

Fixes #3397.

Optimizes Http4sWebSockets.pipeToBody twofold:

1. fast_path If concatenateFragmentedFrames, ignorePong, autoPongOnPing, autoPing flags in WebSocketBodyOutput are all false, lift Http4sWebSocketFrames into REQ, run through pipe, convert RESP back to Http4sWebSocketFrame, use fs2 Chunks for mapping output.
2. standard_path Otherwise, concurrently merge business logic response, autoPings, autoPongOnPing. Use fs2.Channel to perform the merge (more efficient than Stream#mergeHaltL / Stream#parJoin.

Performance results

before:

#[Mean    =       41.509, StdDeviation   =       57.180]
#[Max     =      748.000, Total count    =      6000000]

after standard_path :

#[Mean    =        1.106, StdDeviation   =        0.309]
#[Max     =        4.000, Total count    =      6000000]

after fast_path :

#[Mean    =        0.000, StdDeviation   =        0.013]
#[Max     =        4.000, Total count    =      6000000]

Histograms

tapir_before vs tapir_after:

tapir_after vs plain http4s+blaze:

CPU pressure

Number of async-profiler samples:

• before: 2M
• after standard_path: 734k
• after fast_path: 384k

2M -> 400K means 5x reduction of CPU cycles.

before:

after standard_path:

after fast_path:

Benchmark repo:

https://github.com/kamilkloch/websocket-benchmark/tree/master/results

[plokhotnyuk]

For comparison here is a flame graph for http4s with blaze only (no tapir) under the same load:

@kciesielski @adamw Is there anything left that could defer merging?

[adamw]

@plokhotnyuk the PR was in draft mode, didn't notice it's finalised now. Will take a look on Monday probably :)

[adamw]

I'm not quite sure what's the real difference between the fast & slow tracks - if the flags above are set to appropriate values, apart from some additional initialisation code, the fs2 pipeline seems to end up the same in both cases?

[adamw]

Second question: why these particular flag combination has been chosen for the fast track? The default is to have auto pings (which I think is reasonable), auto pongs, ignore pongs

[kamilkloch]

The difference between the fast and slow track is that a fast track merely pipes the input stream through the business logic, whereas "slow" track involves a heavier machinery of concurrently merging a) business logic output b) autoPing and c) autoPongOnPing streams. This also answers the second question - if we want to avoid concurrent merging of streams for performance reasons, a certain flag combination is enforced, unfortunately. That said, looking at the flags in the fast track:

• concatenateFragmentedFrames is currently a no-op,
• ignorePong and autoPongOnPing can be handled on the http4s level. (autoPongOnPing is a bit of an unfortunate flag as it conflates two distinct behaviors - auto pong AND filtering ping. We cosidered adding WebSocketBodyOutput.ignorePing but ditched the idea due to binary compatibility issues. )
• admittedly, the one remaing is autoPing == false, which has to be handled somewhere else (server layer or busines layer)

Also, even the "slow" track is quite performant compared to the status quo.

[adamw]

Thanks for the reply - so in the "slow path" we are allocating a Channel to do the concurrent merging.

We are using the channel for two purposes: auto-pong and auto-ping. The other flags simply modify the input / output streams, leaving them unchanged if they are false. So maybe we could relax the condition for the fast path, so that it's taken when auto-pong=false & auto-ping=false? I think we should be able to do the concatenating / close-decoding transformations on the fast path as well?

Secondly, I think it would be great to add some documentation, explaining how to best use http4s websockets. As I understand, http4s has also a built-in auto-ping mechanism, which should preferrably be used - that should end up in the docs (a similar note is present in the akka docs, btw.). Similarly, information about the slow/fast paths, what is the performance difference would be great to have, so that people could make an informed choice.

[kamilkloch]

Will revise, cannot promise before Christmas, though :)
Done with the code part.

[kamilkloch]

Regrettably, http4s lacks the auto-ping mechanism and doing it on the Stream (tapir) level turns out to be a (unbelievably) costly operation. Here is why: merging a stream with an empty(!) stream reduces the speed by x800:

val n = 10_000
val m = 100
val s = Stream(1).covary[IO].repeatN(n)
val s2 = s.mergeHaltL(Stream.empty)

s.compile.last.replicateA_(m).timed.map(_._1.toMillis).flatMap(IO.println) >>
s2.compile.last.replicateA_(m).timed.map(_._1.toMillis).flatMap(IO.println)

130
105394

[adamw]

Regrettably, http4s lacks the auto-ping mechanism and doing it on the Stream (tapir) level turns out to be a (unbelievably) costly operation. Here is why: merging a stream with an empty(!) stream reduces the speed by x800:

That looks ... really bad. To be honest I did not suspect that usage of fs2 might dominate I/O, but I guess that's what we're looking at here.

Unless we're using fs2 in a totally wrong way ...

[kamilkloch]

Updated tests (increased number of connections from 10k to 25k). (results for current tapir not included as it failed to finish the test for perf reasons)

Optimizes Http4sWebSockets.pipeToBody twofold:

1. fast_path If autoPongOnPing and autoPing flags in WebSocketBodyOutput are both false, lift Http4sWebSocketFrames into REQ, run through pipe, convert RESP back to Http4sWebSocketFrame, use fs2 Chunks for mapping output.
2. standard_path Otherwise, concurrently merge business logic response, autoPings, autoPongOnPing. Use fs2.Channel to perform the merge (more efficient than Stream#mergeHaltL / Stream#parJoin.

Performance results

http4s + blaze:

#[Mean    =        0.255, StdDeviation   =        0.436]
#[Max     =        2.000, Total count    =     15000000]

after standard_path :

#[Mean    =        0.384, StdDeviation   =        0.486]
#[Max     =        3.000, Total count    =     15000000]

after fast_path :

#[Mean    =       20.424, StdDeviation   =       40.654]
#[Max     =      732.000, Total count    =     15000000]

Histograms

tapir_after_fast_track vs plain http4s+blaze:

tapir_after_fast_track vs tapir_after_slow_track vs plain http4s+blaze:

CPU pressure

Number of async-profiler samples:

• after standard_path: 2M
• after fast_path: 937k
• plain http4s + blaze: 874k

http4s + blaze

after fast_path:

after standard_path:

Benchmark repo:

https://github.com/kamilkloch/websocket-benchmark/tree/master/results

[adamw]

@kamilkloch

Done with the code part.

Great, thanks! :) I can merge & release this now, and we can do the documentation / comments (so that future generation - that is us in a month - will know why the if is there) in a separate PR, if you'd like?

[kamilkloch]

@kamilkloch

Done with the code part.

Great, thanks! :) I can merge & release this now, and we can do the documentation / comments (so that future generation - that is us in a month - will know why the if is there) in a separate PR, if you'd like?

I think it a good idea. By that time we might perhaps land an auto-ping in the http4s/blaze layer, and then we will be abel to add more comprehensive docs, or even change tapir defaults.

[adamw]

Ok, a release is on its way. Please don't forget about the docs ;) I'll be waiting ;)

[kamilkloch]

Ok, a release is on its way. Please don't forget about the docs ;) I'll be waiting ;)

I have not forgotten, waiting until we land a autoPing in http4s, ember and blaze first.