Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FP floating point corner case isn't tested #390

Open
allenjbaum opened this issue Sep 24, 2023 · 1 comment
Open

FP floating point corner case isn't tested #390

allenjbaum opened this issue Sep 24, 2023 · 1 comment

Comments

@allenjbaum
Copy link
Collaborator

There is a strange corner case in FP arithmetic where a result can be normal, but the underflow flag is set.

This occurs for any result value that has exp=0 and mantissa 1.FFFFFF.000000 (through 1.FFFFF.7FFFFF ( I think).
When that result is rounded, ( depending on the rounding mode for the larger values), no rounding happens.
At this point, exponent is zero, so it's a denorm, and (IEEE and RV spec) underflow flag is set
But now the mantissa must be to be converted to a denorm.
The mantissa is shifted by one (so the "hidden bit is no longer hidden), and that shifts out the LSB - which is a "1"
Now it must round again, and adding back rounding bit (again depending on rounding mode) kicks the result back into normalized territory (the smallest normal number)

Multiply doesn't cover this case, and (because it was found in the wild) divide doesn't either.
I have no clue whether add, subtract, FMA, convert, or sqrt tests do, but I suspect not.

These test values must be added to all FP op test suites, single, double and half precision, for all rounding modes. (we don't need to test all possible sign variations, though)

@aamartin0000
Copy link

For reference, see the second Q here:
http://www.jhauser.us/arithmetic/SoftFloat-3/doc/SoftFloat-FAQ.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants