Run benchmarks with current-bench #500
Conversation
|
The link doesn't work (it goes to https://autumn.ocamllabs.io/ocaml-multicore/eio/pull/500; looks like it should be https://autumn.ocamllabs.io/ocaml-multicore/eio/pull/500/base/main). But then it just says "No data for selected interval". |
|
Looks like the benchmarks timed out on the default worker - this tends to happen for multicore benchmarks as the server is configured to run on a single core. @punchagan is configuring this to run on a different server that supports multicore benchmarks. Before it timed out, the status was shown as |
Yes, that's a bug. We've a fix for this that we hope to merge soon. |
|
Making it ready for review as we see the graphs - https://autumn.ocamllabs.io/ocaml-multicore/eio/pull/500/base/main/benchmark/Eio?worker=fermat&image=ocaml%2Fopam%3Adebian-ocaml-5.0. Will look into cleaning them up. |
|
The graphs are a bit hard to read. We should probably remove the "promotions" ones (they're not very useful). |
|
Now it says "No data for selected interval" again. |
|
Seems to be working now. Needs rebasing on |
|
Rebased on |
|
I don't think the rebase worked. Those 32-bit failures were fixed on |
|
Oops, rebased on the right head now! |
talex5
force-pushed
the
current-bench
branch
2 times, most recently
from
8966a8c
to
5bbe6da
Compare
There was a problem hiding this comment.
I squashed your commits into one, and then added a few more:
- It now uses Yojson to generate the JSON, since we're adding a dependency on it anyway.
- I simplified the benchmarks code so the JSON generation is done from main.
- You can now choose to run only one benchmark.
- I tried to improve the labels and descriptions.
- I changed the dune build so that the benchmarks are attached to
eio_main.
If that looks OK to you, we can squash and merge. Thanks!
| dune exec -- ./bench/bench_stream.exe | ||
| dune exec -- ./bench/bench_semaphore.exe | ||
| dune exec -- ./bench/bench_cancel.exe | ||
| if ocamlc -config | grep -q '^system: linux'; then dune exec -- ./lib_eio_linux/tests/bench_noop.exe; fi |
There was a problem hiding this comment.
Might be worth noting that this benchmark got dropped.
|
Are The |
Yes, sorry I forgot to mention that. Paraphrasing what @ElectreAAS told me in a private thread - when we name benchmarks something like Cancel bench is a bit weird indeed. Makes me wonder if |
talex5
force-pushed
the
current-bench
branch
3 times, most recently
from
bc512fb
to
a6fdecd
Compare
Co-authored-by: Thomas Leonard <[email protected]>
Moves the benchmarks to use continuous benchmarking with current-bench. This PR makes benchmark files emit
ocaml-cbcompliant json format results.The code needs more clean-up. Opening this draft early to see how the results get displayed in the current-bench front-end.
I've added
yojsonas a test dependency for now. We can maybe move it to a separate bench opam package.