Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] SparkSQL support TimeTravel #1242

Closed
1 of 3 tasks
Kyofin opened this issue Jun 30, 2022 · 3 comments
Closed
1 of 3 tasks

[Feature Request] SparkSQL support TimeTravel #1242

Kyofin opened this issue Jun 30, 2022 · 3 comments
Labels
enhancement New feature or request

Comments

@Kyofin
Copy link

Kyofin commented Jun 30, 2022

Feature request

Overview

Now we just can use spark dataframe api to query table of history version.

Motivation

Need support use sparksql to time travel.

Further details

just like below:

SELECT * FROM default.tb1  VERSION AS OF 2

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

  • Yes. I can contribute this feature independently.
  • Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
  • No. I cannot contribute this feature at this time.
@Kyofin Kyofin added the enhancement New feature or request label Jun 30, 2022
@zsxwing
Copy link
Member

zsxwing commented Jul 1, 2022

Thanks for raising the request. We already have a ticket for this request #128. You can watch it to get the latest update.

@zsxwing zsxwing closed this as completed Jul 1, 2022
@Kyofin
Copy link
Author

Kyofin commented Jul 1, 2022

Thanks for raising the request. We already have a ticket for this request #128. You can watch it to get the latest update.

I use delta-core_2.12-2.0.0rc1、Spark3.2.1 for testing. Same error happen.

== SQL ==
SELECT * FROM default.people10m_upload TIMESTAMP AS OF '2018-10-18T22:15:12.013Z'
-------------------------------------------------^^^

spark-sql>

@zsxwing
Copy link
Member

zsxwing commented Jul 1, 2022

@huzk8 I updated the issue #128 to make it clear that the SQL syntax would require Spark 3.3 and Delta Lake doesn't support it yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants