-
Notifications
You must be signed in to change notification settings - Fork 244
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEA] hypot SQL function #33
Comments
There are some equivalent forms that avoid overflow/underflow (see https://en.wikipedia.org/wiki/Pythagorean_addition). Will explore. |
To be clear we have not tested for overflow etc in this at all to know what we can/should do. The Spark code is using the java built in https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html#hypot-double-double- Which indicates that it takes care of avoiding intermediate overflow and underflow. I am not sure how critical it is when Spark only supports Doubles as input, but it would be good to know. |
It does look like the naive algorithm is much more prone to overflow because of the squares, I think we can use: x * sqrt(1 + (y/x)^2), where x = max(abs(lhs), abs(rhs)) and y = min(abs(lhs), abs(rhs)) |
Signed-off-by: spark-rapids automation <[email protected]>
Enable retry for parquet write to hive Signed-off-by: Firestarman <[email protected]>
Enable retry for parquet write to hive Signed-off-by: Firestarman <[email protected]>
Enable retry for parquet write to hive Signed-off-by: Firestarman <[email protected]>
Is your feature request related to a problem? Please describe.
would be really great to support the SQl function hypot. We can probably do it initially with just
This might not be good enough with overflows etc and we may need to look a little deeper in how to support it.
The text was updated successfully, but these errors were encountered: