Not arbitrary precision. Floating point range. Lower expectations please.

[pcarrier]

If I cannot get more digits of π or get 1 back this isn't arbitrary precision at all.

>>> 4e10*atan(1)

  40000000000 × atan(1)

   = 31415900000

>>> 10^10^10 + 1 - 10^10^10

  10^(10^10) + (1 - 10^(10^10))

   = 0

Please adjust your README:

Arbitrary-precision numeric type that can handle very large (or small) exponents like 10^(10^10). #

[sharkdp]

The internal precision is greater than what you see in your first example:

≫ atan(1)
  atan(1)
   = 0.785398
≫ atan(1) - 0.78539
  atan(1) - 0.78539
   = 0.0000081634

It's only the output that's currently restricted to six significant digits (see also #107), as this is more than enough for most real-world applications (in my opinion).

insect depends on purscript-quantities, which depends on purescript-decimals, which internally uses decimal.js. The README says that insect uses an arbitrary-precision numeric type because that's what decimal.js advertises. I believe this is only true in the sense that the precision is limited to N significant digits, where N can be configured. For insect, currently, we have N set to 30. But this could be changed (or made configurable: #107).

[pcarrier]

I would still very clearly lower expectations. Not sure what you mean by "internal precision", but I made the tool output a bunch of zeroes that really shouldn't be here in the first place. I can understand truncation, but are you saying that you are showing wrong digits on purpose?

Nothing in the README warns users that the model leads to errors like the examples I gave, and "arbitrary-precision numeric type" definitely led me to the wrong conclusion.

(BTW great tool, I'm already using it a lot)

[sharkdp]

I would still very clearly lower expectations. Not sure what you mean by "internal precision", but I made the tool output a bunch of zeroes that really shouldn't be here in the first place. I can understand truncation, but are you saying that you are showing wrong digits on purpose?

Showing 31415900000 is misleading here, I agree. What I'm saying is that this is only a display issue (which might need solving). The internally stored value has a much higher precision (30 significant digits, instead of 6).

Nothing in the README warns users that the model leads to errors like the examples I gave, and "arbitrary-precision numeric type" definitely led me to the wrong conclusion.

Okay, what do you think:

"Uses a high-precision numeric type with 30 significant digits that handles very large (or small) exponents like 10^(10^10)."

(BTW great tool, I'm already using it a lot)

Thanks! I'm glad you like it.