Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.github: Only test benchmarks, don't run them #2042

Merged
merged 2 commits into from
Apr 10, 2021

Conversation

mxinden
Copy link
Member

@mxinden mxinden commented Apr 10, 2021

Instead of fully executing benchmarks, i.e. running multiple iterations,
each measured and recorded, only test that they compile and run them
with a single execution to make sure they work. The benefit is a reduced
CI runtime.

Flag documentation:

To test that the benchmarks run successfully without performing the
measurement or analysis (eg. in a CI setting), use cargo test --benches.

https://bheisler.github.io/criterion.rs/book/user_guide/command_line_options.html

The above assumes that (a) the benchmark results from CI are likely
noisy and thus (b) no one actually looks at the benchmark results.

Instead of fully executing benchmarks, i.e. running multiple iterations,
each measured and recorded, only test that they compile and run them
with a single execution to make sure they work. The benefit is a reduced
CI runtime.

Flag documentation:

> To test that the benchmarks run successfully without performing the
measurement or analysis (eg. in a CI setting), use cargo test --benches.

https://bheisler.github.io/criterion.rs/book/user_guide/command_line_options.html

The above assumes that (a) the benchmark results from CI are likely
noisy and thus (b) no one actually looks at the benchmark results.
@mxinden mxinden merged commit de261d5 into libp2p:master Apr 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants