Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for variable-precision timestamp type to Hive connector #3977

Closed
martint opened this issue Jun 9, 2020 · 2 comments
Closed

Add support for variable-precision timestamp type to Hive connector #3977

martint opened this issue Jun 9, 2020 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@martint
Copy link
Member

martint commented Jun 9, 2020

Considerations

  • Currently, the connector supports only timestamp(3). The metastore supports "timestamp with optional nanosecond precision" (see https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-TimestampstimestampTimestamps).
    • Each data format may support a different underlying type, so appropriate conversions need to be implemented. Currently, values are truncated to the precision expected by Presto. We should update to do rounding to match the type semantics.
  • There's a performance cliff above timestamp(6), so equating Hive's timestamp to Presto timestamp(9) might not be optimal. We'll probably want a configuration option in the connector to indicate how Hive timestamps should be mapped. We'll also need this for backward compatibility. If existing Hive tables all switch to timestamp(9), may queries may start to fail due to type mismatch issues.
@findepi
Copy link
Member

findepi commented Sep 23, 2020

Read part is generally done by in #4953 and will be available in 342.

@findepi findepi mentioned this issue Sep 23, 2020
9 tasks
@findepi
Copy link
Member

findepi commented Feb 3, 2021

Done by @aalbu and @jirassimok . Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

3 participants