Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support timestamp(n) SQL type #13231

Merged
merged 1 commit into from
Nov 4, 2024
Merged

Support timestamp(n) SQL type #13231

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions datafusion/sql/src/planner.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,8 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
SQLDataType::Char(_)
| SQLDataType::Text
| SQLDataType::String(_) => Ok(DataType::Utf8),
SQLDataType::Timestamp(None, tz_info) => {
SQLDataType::Timestamp(precision, tz_info)
if precision.is_none() || [0, 3, 6, 9].contains(&precision.unwrap()) => {
let tz = if matches!(tz_info, TimezoneInfo::Tz)
|| matches!(tz_info, TimezoneInfo::WithTimeZone)
{
Expand All @@ -466,7 +467,14 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
// Timestamp Without Time zone
None
};
Ok(DataType::Timestamp(TimeUnit::Nanosecond, tz.map(Into::into)))
let precision = match precision {
Copy link
Member

@caicancai caicancai Nov 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I assume this is the standard between vector api and precision?

I am currently working on a similar problem in the process of adapting Arrow in the Calcite community.

Some(0) => TimeUnit::Second,
Some(3) => TimeUnit::Millisecond,
Some(6) => TimeUnit::Microsecond,
None | Some(9) => TimeUnit::Nanosecond,
_ => unreachable!(),
};
Ok(DataType::Timestamp(precision, tz.map(Into::into)))
}
SQLDataType::Date => Ok(DataType::Date32),
SQLDataType::Time(None, tz_info) => {
Expand Down Expand Up @@ -535,8 +543,8 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
| SQLDataType::CharVarying(_)
| SQLDataType::CharacterLargeObject(_)
| SQLDataType::CharLargeObject(_)
// Precision is not supported
| SQLDataType::Timestamp(Some(_), _)
// Unsupported precision
| SQLDataType::Timestamp(_, _)
// Precision is not supported
| SQLDataType::Time(Some(_), _)
| SQLDataType::Dec(_)
Expand Down
35 changes: 35 additions & 0 deletions datafusion/sqllogictest/test_files/timestamps.slt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,41 @@ SELECT COUNT(*) FROM ts_data_secs where ts > from_unixtime(1599566400)
----
2

query P rowsort
SELECT ts FROM ts_data_nanos;
----
2020-09-08T11:42:29.190855123
2020-09-08T12:42:29.190855123
2020-09-08T13:42:29.190855123

query P rowsort
SELECT CAST(ts AS timestamp(0)) FROM ts_data_nanos;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very cool 👍

Copy link
Member

@caicancai caicancai Nov 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The Calcite Arrow adapter will also comply with this standard

----
2020-09-08T11:42:29
2020-09-08T12:42:29
2020-09-08T13:42:29

query P rowsort
SELECT CAST(ts AS timestamp(3)) FROM ts_data_nanos;
----
2020-09-08T11:42:29.190
2020-09-08T12:42:29.190
2020-09-08T13:42:29.190

query P rowsort
SELECT CAST(ts AS timestamp(6)) FROM ts_data_nanos;
----
2020-09-08T11:42:29.190855
2020-09-08T12:42:29.190855
2020-09-08T13:42:29.190855

query P rowsort
SELECT CAST(ts AS timestamp(9)) FROM ts_data_nanos;
----
2020-09-08T11:42:29.190855123
2020-09-08T12:42:29.190855123
2020-09-08T13:42:29.190855123


# count_distinct_timestamps
query P rowsort
Expand Down