Fix division / Better inform about precision

[ad-si]

I would really expect this to provide the correct solution:

$ insect "765537710299 / 500"
1531080000

But it doesn't, correct would be 1531075420.598
How about providing a warning Attention: This calculation might overflow the supported range

... or of course providing the correct solution in the first place 😅
Doesn't look like a too large number size to me.

[sharkdp]

Thank you for your feedback

Please see the FAQ under "What is the internal numerical precision?". Insect calculates the answer up to a precision of 30 digits (try subtracting 1531070000, for example), but only shows 6 significant digits.

This is a somewhat unfortunate behavior right now, but this issue is already tracked id #54 and #107.

[ad-si]

Then you should really remove the "high precision" advertisement from your descriptions. This is not high precision.

And what about the idea to display a warning if the solution is rounded in some way. I think this would be really helpful!

[sharkdp]

Then you should really remove the "high precision" advertisement from your descriptions. This is not high precision.

I disagree. I consider Insect to be a high-precision numerical calculator due to the fact that it performs calculations with 30 digits. It uses a high-precision numeric library under the hood (https://github.com/MikeMcl/decimal.js/). Displaying "just" six significant digits was always enough for me, but I certainly see the need to make that user-configurable (as stated in the linked ticket). However, that is not related to the precision of the calculation itself.

And what about the idea to display a warning if the solution is rounded in some way. I think this would be really helpful!

Insect is (and always will be) a numerical calculator (as opposed to a computer algebra system that performs symbolic calculations). Answers to calculations will always be approximate and there is no way to tell whether the displayed result is the exact result or not. So this is not really feasible. We could use a heuristic. Something like: if the last n digits are zeros (for n larger than, say, 10), consider the first 30 - n digits to be the exact result. However, even in this case, we would have to show that message ("result might be rounded") pretty often. Whenever you have some result whose decimal representation is not finite (something like 1/3).

[ad-si]

Why shouldn't Insect output all digits in the default setting? Then the user can decide how many digits they want to use ^^
I was never surprised by the output of calc, which e.g. easily handles things like calc '3e1000 + 0.1'

It solves the rounding problem by prepending a tilde: calc '1 / 3' => ~0.33333333333333333333

[sharkdp]

I'm going to close this for now. #54 and #107 already deal with this.