Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Superset does not cast trino timestamp correctly #21567

Closed
iercan opened this issue Sep 23, 2022 · 4 comments
Closed

Superset does not cast trino timestamp correctly #21567

iercan opened this issue Sep 23, 2022 · 4 comments
Labels
#bug Bug report data:connect:trino Related to Trino

Comments

@iercan
Copy link
Contributor

iercan commented Sep 23, 2022

I have upgraded sqlalchemy-trino==0.4.1 to trino==0.315.0 since sqlalchemy-trino deprecated . After that we started getting below error

image

I did some investigation and realized superset started to set column type as timestamp(3) instead of timestamp after upgrade. This change prevent superset casts time correctly. I've compared queries and saw that superset does not cast time filter anymore.

Casting before(working)
inserted_at >= from_iso8601_timestamp('2022-09-16T09:07:45.000000')

Casting after(erroneous)
inserted_at >= '2022-09-16 08:47:13.000000'

Clearly there is some compatibility issue with new trino client. For now I solved my problem by downgrading trino client but I'd appreciate If this issue solved so that we can use latest client

How to reproduce the bug

  1. Replace sqlalchemy-trino==0.4.1 with trino==0.315.0
  2. Sync dataset(which have a timestamp column) columns
  3. Send a query with time filter.

Environment

  • superset version: 1.5.0 docker image

Additional context

Related #16223

@iercan iercan added the #bug Bug report label Sep 23, 2022
@mdesmet
Copy link
Contributor

mdesmet commented Oct 6, 2022

This issue is resolved in Superset 1.5.1 (PR link)

See

def convert_dttm(
cls, target_type: str, dttm: datetime, db_extra: Optional[Dict[str, Any]] = None
) -> Optional[str]:
"""
Convert a Python `datetime` object to a SQL expression.
:param target_type: The target type of expression
:param dttm: The datetime object
:param db_extra: The database extra object
:return: The SQL expression
Superset only defines time zone naive `datetime` objects, though this method
handles both time zone naive and aware conversions.
"""
tt = target_type.upper()
if tt == utils.TemporalType.DATE:
return f"""DATE '{dttm.date().isoformat()}'"""
if tt in (
utils.TemporalType.TIMESTAMP,
utils.TemporalType.TIMESTAMP_WITH_TIME_ZONE,
):
return f"""TIMESTAMP '{dttm.isoformat(timespec="microseconds", sep=" ")}'"""
return None

Let me know if that works for you!

@iercan
Copy link
Contributor Author

iercan commented Oct 6, 2022

@mdesmet I've tested with 1.5.2. Looks like problem still exists.

Btw we are using mongo connector if it helps.

image

@mdesmet
Copy link
Contributor

mdesmet commented Oct 7, 2022

I'm able to reproduce your issue. will post a PR later today.

@rusackas
Copy link
Member

Seems like this should have been closed by the linked PR. Holler if there's any need to re-open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
#bug Bug report data:connect:trino Related to Trino
Projects
None yet
Development

No branches or pull requests

3 participants