-
Notifications
You must be signed in to change notification settings - Fork 6.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hyperbolic function returns incorrect result #9460
Comments
@Michael-Daugherty can you take a screenshot? |
I know 20 digits of precision is stupid. But that's not the point. Something is wrong with the algorithm causing this discontinuous result. Possibly, truncate the input at 19 before doing the trig function. I'm a mechanical engineer, not a computer scientist or mathematician. |
it is a floating point precision issue. We don't control the calc library and just blindly send data to it. Once the precision gets above a precision, you get what is basically a casting error. Betting it casts to float vs double. When we shift to the calculator's engine, this (famous last words) should be fixed. |
#2265 is the tracking item for calc engine |
ℹ Computer information
📝 Provide detailed reproduction steps:
Alt Space sinh(.9999999999999999999)
✔️ Expected result:
1.175201193
❌ Actual result:
0.0038758297
The text was updated successfully, but these errors were encountered: