You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As noted, the std-dev allows us to easier differentiate benchmarks like the navy vs orange ones from this analysis (one has very high variance and the other is very 'stable').
We can see how the stdev allows us to better judge the benchmarks - for example the high stdev in the Enso variants may show that the warmup time is insufficient and should be made larger if we want to see the peak performance.
Our benchmark charts currently displays only score which is an average of one benchmark iteration in milliseconds.
Since JMH can, and does, output stddev, it would be nice to:
Tasks
stddev is a very important metric as it tells us basically how much we can trust the results for a particular benchmark.
The text was updated successfully, but these errors were encountered: