Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support array with scalar arithmetic operation for decimal data type #2233

Merged

Conversation

liukun4515
Copy link
Contributor

@liukun4515 liukun4515 commented Apr 14, 2022

Which issue does this PR close?

part of #122

schema:
CREATE EXTERNAL TABLE food (a DECIMAL(10,5), b decimal(20,15), c boolean) xxxx;

This query select a+b from table was supported, but the query select a+10 from table is not supported.

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added the datafusion Changes in the datafusion crate label Apr 14, 2022
@liukun4515 liukun4515 marked this pull request as ready for review April 16, 2022 12:42
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @liukun4515

@@ -280,25 +280,64 @@ where
.collect()
}

fn arith_decimal_scalar<F>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to move some / all of this decimal manipulation into the arrow-rs crate at some point

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your reminder and I have a plan to do this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But now I find a bug in the arithmetic operation. https://github.com/apache/arrow-datafusion/blob/c91efc27658e58264c4f346a5cfdec8810179e90/datafusion/physical-expr/src/expressions/binary.rs#L302 and https://github.com/apache/arrow-datafusion/blob/c91efc27658e58264c4f346a5cfdec8810179e90/datafusion/physical-expr/src/expressions/binary.rs#L307

In the divide method, we may lose of precision by converting the i128 to f64.
We use the f64 to store the larger range of data, but we lose the precision.

I will try to figure out this issue and resolve this.

@liukun4515
Copy link
Contributor Author

I think It can be merged

@alamb
Copy link
Contributor

alamb commented Apr 19, 2022

I think It can be merged

Sounds good to me -- feel free to do it yourself next time; I normally wait for at least one approving review from a maintainer or active contributor. If the change is trivial I'll merge it then and if a little more involved I try and leave it open for 24 hours.

@alamb alamb merged commit 7334481 into apache:master Apr 19, 2022
@alamb
Copy link
Contributor

alamb commented Apr 19, 2022

Thanks again @liukun4515

@liukun4515 liukun4515 deleted the support_arithmetic_array_scalar_decimal branch April 20, 2022 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants