Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable casting from Date64 to Timestamp #3038

Merged
merged 1 commit into from
Nov 7, 2022

Conversation

gruuya
Copy link
Contributor

@gruuya gruuya commented Nov 7, 2022

Which issue does this PR close?

Closes #3037

Rationale for this change

I'm working on adding support for remote tables in Seafowl (in analogy with PG FDWs) using the connector-x crate. An issue that comes up is the fact that connector-x uses arrow's Date64 as the target type for coercion of various datetime/timestamp types from other DBs.

More specifically, Postgres TIMESTAMP is converted into Date64. On the other hand, Datafusion interprets TIMESTAMP type as arrow's Timestamp(TimeUnit::Nanosecond, None). Consequently, I end up needing to cast from Date64 (as returned by connector-x) into Timestamp (as expected by Datafusion), but such cast is not supported at the moment.

What changes are included in this PR?

I expanded the cast kernel to facilitate the Date64 -> Timestamp cast, as well as the can_cast_types function to reflect this.

Are there any user-facing changes?

Trying to cast from Date64 to Timestamp will not error out after this; not sure whether this warrants a doc change somewhere.

@github-actions github-actions bot added the arrow Changes to the arrow crate label Nov 7, 2022
@gruuya gruuya force-pushed the date64-to-timestamp-cast branch from 4f72550 to a8b0737 Compare November 7, 2022 06:31
Copy link
Contributor

@tustvold tustvold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@tustvold tustvold merged commit 951caed into apache:master Nov 7, 2022
@ursabot
Copy link

ursabot commented Nov 7, 2022

Benchmark runs are scheduled for baseline = 4df1cc4 and contender = 951caed. 951caed is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support casting Date64 to Timestamp
4 participants