Limit decimal places in JSON output of averages #161
Conversation
As nlohmann::json always attempts to maintain full precision when converting from a C++ float/double to a JSON number it prints way too many decimal places in many cases. Use std::modf to split the double precision value into integer and fractional pieces and then truncate the DPs if needed. Note that it's quite tricky to do this, so the fractional piece is stored as a volatile to prevent the 'int(value*N)/double(N)' from being optimised away. Note also that this is quite an expensive operation, but in the context of prmon we can afford this at the typical monitoring cadence.
There was a problem hiding this comment.
Thanks a lot for this @graeme-a-stewart. Maybe this is splitting hairs but I don't think we're strictly limiting our measurement to a per-mille level accuracy here, no? What we're instead doing is that anything larger than 1k will be converted to an integer, while anything less than 1k will have three decimal places. E.g. 123456.1236 will be 123456 and 123.1236 will be 123.123, where the latter also suffers a bias from the lack of rounding (which would affect the first, too, of course). They both imply precision that is better than per-mille and we're not doing any rounding correction etc.
Having said that, I think what we have is good enough for our purposes. It's just that we should perhaps advertise this a bit clearer, unless I'm missing something.
Please let me know, many thanks.
|
Hi @amete - yes, you're right, it's printing a rounded integer for > 1000 and truncating to 3DP for <1000. I wanted to avoid complexities like calculating If you also agree this is good enough I will update the description to be more accurate of the actual method. |
I completely agree this is good enough for all practical purposes. Updating the description sounds like a good idea, just to make sure there are no misunderstandings. Many thanks. |
As nlohmann::json always attempts to maintain full precision
when converting from a C++ float/double to a JSON number
it prints way too many decimal places in many cases.
Use std::modf to split the double precision value into integer
and fractional pieces and then truncate the DPs if needed.
Note that it's quite tricky to do this, so the fractional piece is
stored as a volatile to prevent the 'int(value*N)/double(N)' from
being optimised away. Note also that this is quite an expensive
operation, but in the context of prmon we can afford this at the
typical monitoring cadence.
Closes #158