Skip to content
This repository has been archived by the owner on Sep 14, 2023. It is now read-only.

Port Python benchmark to pytest-benchmark #18

Closed
mre opened this issue Jul 4, 2018 · 4 comments
Closed

Port Python benchmark to pytest-benchmark #18

mre opened this issue Jul 4, 2018 · 4 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@mre
Copy link
Owner

mre commented Jul 4, 2018

Our current Python benchmarks are based on ujson's script here.
It's a bit clunky to use and not really idiomatic (redundant code, custom implementation for printing the results as a table).
In addition to that, it's hard to make code changes and see the performance impact, because the measurements fluctuate a lot from run to run.

A modern alternative is pytest-benchmark. Since we already use pytest, it's would fit in well with the rest of our testing pipeline.
It provides powerful output methods for different use-cases:

grafik

grafik

It can also plot histograms to visualize the error bars as a box-plot.

grafik

It would be nice to move to pytest-benchmark as we could compare the measurements after each change using Travis CI.

@mre mre added enhancement New feature or request good first issue Good for newcomers labels Jul 4, 2018
@wdv4758h
Copy link
Contributor

wdv4758h commented Jul 6, 2018

Not directly related to benchmark, I find out pytest-sugar shows pretty sweat test progress and start using it this year. Just FYI, wonder if you guys have interest 😃

pytest-sugar1
pytest-sugar2
pytest-sugar3

@mre
Copy link
Owner Author

mre commented Jul 6, 2018

@wdv4758h sounds like a cool thing to have. If you like, go ahead and create a PR for it. 👍 🍭

@mre
Copy link
Owner Author

mre commented Jul 7, 2018

Initial work is done by @RSabet in #19.

@mre
Copy link
Owner Author

mre commented Jul 7, 2018

Closing this as #19 is done now. @wdv4758h a pull request for pytest-sugar is appreciated if you find the time. 😃

@mre mre closed this as completed Jul 7, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants