Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support mathematics operation for decimal data type #1554

Merged
merged 9 commits into from
Jan 18, 2022

Conversation

liukun4515
Copy link
Contributor

Which issue does this PR close?

part of #122

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added ballista datafusion Changes in the datafusion crate labels Jan 13, 2022
@liukun4515
Copy link
Contributor Author

will rebase the master after #1483 merged.

@liukun4515 liukun4515 force-pushed the mathematics_op_decimal branch from 2bc413a to bd8a149 Compare January 17, 2022 03:17
@liukun4515 liukun4515 force-pushed the mathematics_op_decimal branch from bd8a149 to 3d15e8f Compare January 17, 2022 04:30
@liukun4515 liukun4515 marked this pull request as ready for review January 17, 2022 06:32
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @liukun4515 -- this is looking good. The only concern I have is the new split between numerical_coertion and mathematics_numerical_coercion

@@ -162,12 +162,141 @@ fn get_comparison_common_decimal_type(
}
}

// Convert the numeric data type to the decimal data type.
// Now, we just support the signed integer type and floating-point type.
fn convert_numeric_type_to_decimal(numeric_type: &DataType) -> Option<DataType> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this would be better called coerce_numeric_type_to_decimal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

bool_builder.append_value(true)?;
} else {
bool_builder.append_value(left.value(i) != right.value(i))?;
match (left.is_null(i), right.is_null(i)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

}
}

fn mathematics_numerical_coercion(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function looks almost the same as numerical_coercion -- which is used for equality

What is the reason for making a new function rather than extending numerical_coercion with Decimal support?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image
The numerical_coercion is used by other coercion functions:

  1. numerical_coercion->eq_coercion -> Operator::IsDistinctFrom | Operator::IsNotDistinctFrom =>
  2. numerical_coercion -> dictionary_value_coercion
    If I extend the numerical_coercion, it may take a side effect to other coercion logical.
    I will refactor and merge these two together and make it clear if there is no conflict.
    @alamb

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you try extending numerical_coercion (and if so, did it have any side effects)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb I will extend it in the follow-up pull request and support dict and distinct with decimal.
In the follow-up pull request, I think we can refactor out a common part.

@liukun4515 liukun4515 requested a review from alamb January 17, 2022 15:25
@alamb alamb merged commit c549d51 into apache:master Jan 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants