feat(@libp2p/protocol-perf): Implement perf protocol

[MarcoPolo]

Implements libp2p/specs#478

[mxinden]

🚀

[BigLep]

What are the next steps on this? I'd like to make sure js-libp2p isn't getting left behind on the perf dashboarding.

[wemeetagain]

The logic and stylistic concerns lgtm

[maschad]

What are the next steps on this? I'd like to make sure js-libp2p isn't getting left behind on the perf dashboarding.

Agreed @BigLep . I am going to refactor the PerfService to allow this to be configured the way configure our other optional services now following @achingbrain 's suggestion, as well as incorporate @wemeetagain 's feedback and then that should return this PR to a ready state.

[mxinden]

Thank you @maschad.

and then that should return this PR to a ready state.

I will then integrate it into libp2p/test-plans#163.

[maschad]

CI check failure is fixed by #1801

[maschad]

Since this is our first benchmark tests there isn't really a precedent, but do we want to run these types of tests in our regular PR workflow or would it be better suited to be run on the main branch ? cc @achingbrain @wemeetagain @MarcoPolo

[mxinden]

What are you referring to with "these types of tests"? For libp2p/test-plans#184 we plan to run it before every release for now.

[maschad]

What are you referring to with "these types of tests"? For libp2p/test-plans#184 we plan to run it before every release for now.

I was referring to benchmark tests there (as opposed to our other types of tests such as unit or integration tests). But if the plan is to run it before every release then the answer to my question would be to run these post-merge on the main branch.

[MarcoPolo]

In the ideal world, this should run on every PR to highlight any perf regressions or improvements. It should fail CI if perf degrades by more than X% relative to the main branch.

[p-shahi]

Can we pull in https://github.com/ipfs-shipyard/js-libp2p-transfer-performance into test/perf as a followup? This would allow us to close libp2p/test-plans#65

[p-shahi]

@achingbrain suggestion: add this as a package to the monorepo (in a folder called perf/ in packages/)

[maschad]

Currently blocked by a huge refactor in #1833

[maschad]

Update:

Running these perf tests locally i.e. running npm run start with a server and dialler will output a latency measurement to stdout.

I had spent a significant amount of time trying to run the perf tests using the terraform provisioned infrastructure but to no avail, it ended up being a lot of AWS wrangling which I think is tangential to the essence of this PR, more is outlined here

These tests will be ran on this workflow but given this PR outputs the measurement of latency to stdout as required by the test plans implementation setup this PR should be ready for review.

[achingbrain]

Looking promising, comments inline.

[MarcoPolo]

Can only comment. I'd request changes if I could.

I think we can get rid of some of the code here 😄

[maschad]

I've made the requested changes and I've removed the perf workflow as discussed here for now, once libp2p/test-plans#244 is merged we can introduce a workflow to run on release / main in a follow up PRs.

[mxinden]

@maschad please ping me once you need another review here.

[MarcoPolo]

I would approve if I could (my own PR).

[achingbrain]

LGTM

Just a few changes before merge:

1. Update the @packageDocumentation in index.ts to show examples for perf not circuit relay - this will appear on the API docs page for the package
2. Rename the packages/perf folder to packages/protocol-perf - to be consistent with peer-discovery-mdns/peer-discovery-bootstrap, transport-tcp/transport-webrtc/transport-websockets/etc