proposal: benchmarks or benchstat: should question timings less than 1ns

[bboreham]

From time to time I see someone quote a benchmark output with a timing like "0.03995ns".

Invariably this means the benchmark is doing nothing at all, e.g. because they forgot to use b.N.

Similarly to when benchstat advises to use more iterations to get a stable result, it could advise that such a short timing may be an error.

[ianlancetaylor]

CC @aclements

[rsc]

I had an issue somewhere, which I can't find right now, to report when a benchmark seems to be ignoring b.N. Diagnosing that in package testing seems like it makes more sense than doing it in benchstat.

[ianlancetaylor]

Perhaps #38677 ?

[rsc]

CL 230978 is what I was thinking of. I think it would catch anything that would produce 0.03995ns.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Austin pointed out various problems with my CL so I wrote a vet check instead. This is now probably a duplicate of #38677.

[bboreham]

One case I don’t think would be covered is when the compiler optimizes away your entire loop.

[rsc]

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group