Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add new types to udaf functions #8081

Merged
merged 4 commits into from
Sep 1, 2021
Merged

fix: add new types to udaf functions #8081

merged 4 commits into from
Sep 1, 2021

Conversation

jzaralim
Copy link
Contributor

@jzaralim jzaralim commented Aug 31, 2021

Description

Adds support for timestamp, time, date and bytes to collect_list, collect_set, latest_by_offset and earliest_by_offset.

Fixes #7787

Testing done

unit/qtt

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@jzaralim jzaralim requested a review from a team as a code owner August 31, 2021 00:13
Copy link
Contributor

@vcrfxia vcrfxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jzaralim ! LGTM with a couple minor notes inline.

return latest(STRUCT_TIMESTAMP, ignoreNulls, getComparator(ignoreNulls));
}

@UdafFactory(description = "return the latest N value of a timestamp column",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
@UdafFactory(description = "return the latest N value of a timestamp column",
@UdafFactory(description = "return the latest N values of a timestamp column",

and similar for the other descriptions in this file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

},
{
"name": "collect_list bytes map",
"format": ["JSON"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, does the PROTOBUF format not support the BYTES type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does, it just requires specifying the schema to the test, which I was too lazy to do.

Zara Lim added 2 commits August 31, 2021 12:19
@jzaralim jzaralim merged commit a3ea6a4 into master Sep 1, 2021
@jzaralim jzaralim deleted the udaf-types branch September 1, 2021 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EARLIEST_BY_OFFSET not supporting timestamp type
2 participants