Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement serialization for UDWF and UDAF in plan protobuf #6769

Merged
merged 3 commits into from
Jun 30, 2023

Conversation

parkma99
Copy link
Contributor

Which issue does this PR close?

Closes #4584 #6733

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@parkma99
Copy link
Contributor Author

cc @alamb I am not sure this is good implement?😊

@tustvold tustvold changed the base branch from main to backup-main June 27, 2023 11:11
@tustvold tustvold changed the base branch from backup-main to main June 27, 2023 11:11
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good to me -- thank you @parkma99 🤗

row_number_frame,
));
ctx.register_udaf(dummy_agg);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

THis is great!

Now all we need is a test for the WindowUDF (aka WindowFunction::WindowUDF) and I think this PR is ready to go!

@@ -585,16 +585,16 @@ impl TryFrom<&Expr> for protobuf::LogicalExprNode {
)
}
// TODO: Tracked in https://github.com/apache/arrow-datafusion/issues/4584
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can remove the TODO comments as well

@parkma99
Copy link
Contributor Author

Thanks @alamb , I will update the test case lately.😄

@parkma99 parkma99 marked this pull request as ready for review June 29, 2023 13:53
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @parkma99 -- this is looking really close. I have just one more request on the tests 🙏

values: &[ArrayRef],
range: &std::ops::Range<usize>,
) -> Result<ScalarValue> {
// Again, the input argument is an array of floating
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can just use todo!() here and in the other functions? These should not be called during the logical plan testing

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @parkma99 -- looks great

@alamb alamb merged commit bfffdba into apache:main Jun 30, 2023
@parkma99 parkma99 deleted the serialize_udaf_and_udwf branch June 30, 2023 10:45
2010YOUY01 pushed a commit to 2010YOUY01/arrow-datafusion that referenced this pull request Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement serialization for UDAF as window function in plan protobuf
2 participants