Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

strptime function definition violates spec #487

Closed
vbarua opened this issue Apr 4, 2023 · 8 comments
Closed

strptime function definition violates spec #487

vbarua opened this issue Apr 4, 2023 · 8 comments
Assignees

Comments

@vbarua
Copy link
Member

vbarua commented Apr 4, 2023

Context

While upgrading substrait-java to the latest version of the spec, tests failed due to multiple scalar functions being added w/ the same type signature.

As defined, strptime

name: "strptime"
description: >-
Parse string into timestamp/date/time using provided format,
see https://man7.org/linux/man-pages/man3/strptime.3.html for reference.
If timezone is present in timestamp and provided as parameter an error is thrown.
Timezone strings must be as defined by IANA timezone database (https://www.iana.org/time-zones).
Examples: "Pacific/Marquesas", "Etc/GMT+1".
If timezone is supplied as parameter and present in the parsed string the parsed timezone is used.
If parameter supplied timezone is invalid an error is thrown.
impls:
- args:
- name: timestamp_string
value: string
- name: format
value: string
- name: timezone
description: Timezone string from IANA tzdb.
value: string
return: timestamp_tz
- args:
- name: timestamp_string
value: string
- name: format
value: string
return: timestamp_tz
- args:
- name: date_string
value: string
- name: format
value: string
return: date
- args:
- name: time_string
value: string
- name: format
value: string
return: time

has the following signatures

* strptime:str_str_str
* strptime:str_str
* strptime:str_str
* strptime:str_str

Problem

The addition of the the strptime function in #272 violates the spec.

Specifically

It is required that two function implementations with the same simple name must resolve to different compound names using types. If two function implementations in a YAML file resolve to the same compound name, the YAML file is invalid and behavior is undefined.

https://substrait.io/extensions/#function-signature-compound-names

Changes were introduced by #272

@rok
Copy link
Contributor

rok commented Apr 5, 2023

Oh I missed this. How about: strptime -> strptime_timestamp, strptime_date, strptime_time?

@richtia
Copy link
Member

richtia commented Apr 5, 2023

Oh I missed this. How about: strptime -> strptime_timestamp, strptime_date, strptime_time?

This is what I was thinking as well...separating the function into 3 separate ones. I'm curious how producers/consumers will handle this with regards to mapping though.

@rok
Copy link
Contributor

rok commented Apr 5, 2023

I'm curious how producers/consumers will handle this with regards to mapping though.

What would you expect to be an issue there?

@richtia
Copy link
Member

richtia commented Apr 5, 2023

What would you expect to be an issue there?

Not necessarily expecting any issues. Just wondering how it would work since I've typically seen a single strptime function. I guess all 3 from substrait would just map the 1? Any thoughts @westonpace?

@richtia
Copy link
Member

richtia commented Apr 10, 2023

@rok I went ahead and submitted a PR for this since I have a few other PRs waiting for the fix.

@rok
Copy link
Contributor

rok commented Apr 10, 2023

Thanks @richtia !

@richtia
Copy link
Member

richtia commented Apr 10, 2023

Thanks @richtia !

Np! Do you mind giving it a quick look? #493

@richtia richtia self-assigned this Apr 10, 2023
westonpace pushed a commit that referenced this issue Apr 11, 2023
Fix spec violation described in
#487 by
separating `strptime` into `strptime_timestamp`, `strptime_date`, and
`strptime_time`

Here's the original PR adding this in case anyone wants to see the
discussion
surrounding this addition:
#272
@richtia
Copy link
Member

richtia commented Apr 11, 2023

Should be resolved by #493 now

@richtia richtia closed this as completed Apr 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants