-
Notifications
You must be signed in to change notification settings - Fork 864
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
isNan and isInf in Constant.cpp produce wrong results on big-endian systems #2802
Comments
There were two implementations of isInf() and isNan(), in Constant.cpp and in intermOut.cpp. The former only works on little-endian systems, the latter is a wrapper for library functions and works regardless of endianness. Move the second version into Common.h and adopt it in both places. Thereby avoid the duplication and fix for big-endian systems. On s390x, this fixes the test case Glsl/CompileToAstTest.FromFile/constFold_frag. TODO regression test on x86. Fixes KhronosGroup#2802
There were two implementations of isInf() and isNan(), in Constant.cpp and in intermOut.cpp. The former only works on little-endian systems, the latter is a wrapper for library functions and works regardless of endianness. Move the second version into Common.h and adopt it in both places. Thereby avoid the duplication and fix for big-endian systems. On s390x, this fixes the test case Glsl/CompileToAstTest.FromFile/constFold_frag. Fixes KhronosGroup#2802
I believe that the use of the union exercised undefined behaviour. It's a type-based alias analysis violation. Thanks for fixing this! |
There were two implementations of isInf() and isNan(), in Constant.cpp and in intermOut.cpp. The former only works on little-endian systems, the latter is a wrapper for library functions and works regardless of endianness. Move the second version into Common.h and adopt it in both places. Thereby avoid the duplication and fix for big-endian systems. On s390x, this fixes the test case Glsl/CompileToAstTest.FromFile/constFold_frag. TODO regression test on x86. TODO build-test on Windows!! Fixes KhronosGroup#2802
In my patch, I missed moving the |
…ing (updated) There were two implementations of isInf() and isNan(), in Constant.cpp and in intermOut.cpp. The former only works on little-endian systems, the latter is a wrapper for library functions and works regardless of endianness. Move the second version into Common.h and adopt it in both places. Thereby avoid the duplication and fix for big-endian systems. A previous commit with the same intent and purpose had missed a required header for builds on Windows. On s390x, this fixes the test case Glsl/CompileToAstTest.FromFile/constFold_frag. Fixes KhronosGroup#2802
There were two implementations of isInf() and isNan(), in Constant.cpp and in intermOut.cpp. The former only works on little-endian systems, the latter is a wrapper for library functions and works regardless of endianness. Move the second version into Common.h and adopt it in both places. Thereby avoid the duplication and fix for big-endian systems. On s390x, this fixes the test case Glsl/CompileToAstTest.FromFile/constFold_frag. Fixes KhronosGroup#2802
…ing (updated) There were two implementations of isInf() and isNan(), in Constant.cpp and in intermOut.cpp. The former only works on little-endian systems, the latter is a wrapper for library functions and works regardless of endianness. Move the second version into Common.h and adopt it in both places. Thereby avoid the duplication and fix for big-endian systems. A previous commit with the same intent and purpose had missed a required header for builds on Windows. On s390x, this fixes the test case Glsl/CompileToAstTest.FromFile/constFold_frag. Fixes KhronosGroup#2802
Both attempt to decode a
double
into high and low 32-bitint
s via a union:glslang/glslang/MachineIndependent/Constant.cpp
Lines 49 to 52 in 8f56d73
On little-endian systems that works as intended. On big-endian systems, such as s390x, the 32-bit ints end up in reversed order and thus the two functions fail to detect Infs or NaNs.
Reproduction:
glslangValidator -C -i Test/constFold.frag
and compareout10
andout11
to expected values -- on s390x, they are all 0 since none of the NaN or Inf conditions are detected correctly.The text was updated successfully, but these errors were encountered: