Issue using type datetime or time

[nypgand1]

Make sure these boxes are checked before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if any
• I have reproduced the issue with at least the latest released version of superset
• I have checked the issue tracker for the same issue and I haven't found one similar

Superset version

superset==0.14.1
psycopg2==2.6.2
python-3.4.1 on heroku
heroku-postgresql 9.6.1

Actual results

If I use type datetime field and query, I got an error that says datetime can not be understood by numpy. Maybe it cause by type timestamp in postgresql?

Traceback (most recent call last): 
    File "/app/.heroku/python/lib/python3.4/site-packages/superset/views.py", line 2393, in sql_json 
      data = sql_lab.get_sql_results(query_id, return_results=True) 
    File "/app/.heroku/python/lib/python3.4/site-packages/celery/local.py", line 188, in __call__ 
      return self._get_current_object()(*a, **kw) 
    File "/app/.heroku/python/lib/python3.4/site-packages/celery/app/task.py", line 420, in __call__ 
      return self.run(*args, **kwargs) 
    File "/app/.heroku/python/lib/python3.4/site-packages/superset/sql_lab.py", line 145, in get_sql_results 
      payload['columns'] = cdf.columns_dict if cdf else [] 
    File "/app/.heroku/python/lib/python3.4/site-packages/superset/dataframe.py", line 54, in columns_dict 
      agg = agg_func(self.__df.dtypes[col], col) 
Jan 04 20:03:50  heroku/router:  at=info method=POST path="/superset/sql_json/" host=.herokuapp.com request_id=879d077e-4496-41c7-8af3-5549927db6cf fwd="119.14.81.136" dyno=web.1 connect=1ms service=598ms status=500 bytes=208 
    File "/app/.heroku/python/lib/python3.4/site-packages/superset/dataframe.py", line 110, in agg_func 
      if np.issubdtype(dtype, np.number): 
    File "/app/.heroku/python/lib/python3.4/site-packages/numpy/core/numerictypes.py", line 755, in issubdtype 
      return issubclass(dtype(arg1).type, arg2) 
  TypeError: data type not understood 

If I split it into 2 fields (type date & time) and query, I got an error that says datetime.time object can not be serializable. Cause in method json_iso_dttm_ser(), it deals with type datetime or date only, so i think using type timedelta (type interval in postgresql) will be the same case.

Traceback (most recent call last): 
  File "/app/.heroku/python/lib/python3.4/site-packages/superset/views.py", line 2393, in sql_json 
    data = sql_lab.get_sql_results(query_id, return_results=True) 
  File "/app/.heroku/python/lib/python3.4/site-packages/celery/local.py", line 188, in __call__ 
    return self._get_current_object()(*a, **kw) 
  File "/app/.heroku/python/lib/python3.4/site-packages/celery/app/task.py", line 420, in __call__ 
    return self.run(*args, **kwargs) 
  File "/app/.heroku/python/lib/python3.4/site-packages/superset/sql_lab.py", line 147, in get_sql_results 
    payload = json.dumps(payload, default=utils.json_iso_dttm_ser) 
  File "/app/.heroku/python/lib/python3.4/json/__init__.py", line 237, in dumps 
    **kw).encode(obj) 
  File "/app/.heroku/python/lib/python3.4/json/encoder.py", line 192, in encode 
    chunks = self.iterencode(o, _one_shot=True) 
  File "/app/.heroku/python/lib/python3.4/json/encoder.py", line 250, in iterencode 
    return _iterencode(o, 0) 
  File "/app/.heroku/python/lib/python3.4/site-packages/superset/utils.py", line 244, in json_iso_dttm_ser 
    "Unserializable object {} of type {}".format(obj, type(obj)) 
TypeError: Unserializable object 17:00:00 of type <class 'datetime.time'> 

[xrmx]

The second traceback should be handled by this:

diff --git a/superset/utils.py b/superset/utils.py
index 9239b71..5133f34 100644
--- a/superset/utils.py
+++ b/superset/utils.py
@@ -5,7 +5,7 @@ from __future__ import print_function
 from __future__ import unicode_literals
 
 from builtins import object
-from datetime import date, datetime
+from datetime import date, datetime, time
 import decimal
 import functools
 import json
@@ -239,6 +239,8 @@ def json_iso_dttm_ser(obj):
         obj = obj.isoformat()
     elif isinstance(obj, date):
         obj = obj.isoformat()
+    elif isinstance(obj, time):
+        obj = obj.isoformat()
     else:
         raise TypeError(
             "Unserializable object {} of type {}".format(obj, type(obj))

[xrmx]

@nypgand1 latest master should work for you for the second backtrace, regarding the first one if you can add some prints and sort out what arg1 and arg2 it would be helpful. Let us know!

[nypgand1]

@xrmx Thank you so much!

[ama-ableton]

As noted in #1929, i get the same error as the first traceback above. Appears that postgres has two different timestamp column types TIMESTAMP WITH TIME ZONE, which I get an error with, and TIMESTAMP WITHOUT TIME ZONE which works fine with superset.

Here's the arg1 and arg2 passed into the issubdtype function mentioned in the stack trace:

arg1: datetime64[ns, psycopg2.tz.FixedOffsetTimezone(offset=0, name=None)]
arg2: <type 'numpy.number'>

I just tried selecting only the timestamp column, and got a different trace:

2017-01-09 18:03:05,800:ERROR:root:izip argument #2 must support iteration
Traceback (most recent call last):
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/superset/views.py", line 2500, in sql_json
    data = sql_lab.get_sql_results(query_id, return_results=True)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/celery/local.py", line 188, in __call__
    return self._get_current_object()(*a, **kw)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/celery/app/task.py", line 420, in __call__
    return self.run(*args, **kwargs)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/superset/sql_lab.py", line 170, in get_sql_results
    payload['data'] = cdf.data if cdf else []
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/superset/dataframe.py", line 31, in data
    return self.__df.to_dict(orient='records')
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/pandas/util/decorators.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/pandas/core/frame.py", line 857, in to_dict
    for row in self.values]
TypeError: izip argument #2 must support iteration

[btzsoft]

Anyone have any idea how to fix this issue with TIMESTAMP WITH TIME ZONE.
Thanks

[mistercrunch]

Closing, should be fixed by #1907

[mschulkind]

@mistercrunch Should we open a second bug for the first stack trace? It is still affecting me.

Traceback (most recent call last):
  File "/home/ubuntu/superset_git/superset/superset/views.py", line 2575, in sql_json
    data = sql_lab.get_sql_results(query_id, return_results=True)
  File "/home/ubuntu/superset_git/venv/lib/python3.4/site-packages/celery-3.1.23-py3.4.egg/celery/local.py", line 188, in __call__
    return self._get_current_object()(*a, **kw)
  File "/home/ubuntu/superset_git/venv/lib/python3.4/site-packages/celery-3.1.23-py3.4.egg/celery/app/task.py", line 420, in __call__
    return self.run(*args, **kwargs)
  File "/home/ubuntu/superset_git/superset/superset/sql_lab.py", line 150, in get_sql_results
    payload['columns'] = cdf.columns_dict if cdf else []
  File "/home/ubuntu/superset_git/superset/superset/dataframe.py", line 54, in columns_dict
    agg = agg_func(self.__df.dtypes[col], col)
  File "/home/ubuntu/superset_git/superset/superset/dataframe.py", line 110, in agg_func
    if np.issubdtype(dtype, np.number):
  File "/home/ubuntu/superset_git/venv/lib/python3.4/site-packages/numpy-1.12.0-py3.4-linux-x86_64.egg/numpy/core/numerictypes.py", line 755, in issubdtype
    return issubclass(dtype(arg1).type, arg2)
TypeError: data type not understood

This is using a very recent version built from git:

$ git rev-parse  HEAD
2ace73e9a1f1cf1a20eff632d08ea440f5f88607

[xrmx]

@mschulkind per #1900 (comment) it's not the stacktrace that is needed but the actual types. Are yours the very same of #1900 (comment) ?

[mschulkind]

Yes, it is exactly the same type. Looks like this is an even more specific bug that could be reopened:
#1929