Negative timedeltas for deltas > 292 years

[MMCMA]

Code Sample, a copy-pastable example if possible

import pandas as pd
dates = pd.Series(pd.date_range('1700-01-01', periods=4500, freq='m'))
days_delta = (dates-dates.min()).astype('timedelta64[D]')

Expected Output

Should return only positive timedeltas, however for deltas greater than 292 years the values are negative.

output of pd.show_versions()

commit: None
python: 3.4.4.final.0
python-bits: 64
OS: Windows
OS-release: 8.1
machine: AMD64
processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.17.1
nose: None
pip: 8.0.3
setuptools: 20.1.1
Cython: 0.23.4
numpy: 1.10.4
scipy: 0.17.0
statsmodels: 0.6.1
IPython: 4.0.3
sphinx: 1.3.5
patsy: 0.4.1
dateutil: 2.5.0
pytz: 2015.7
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.4.6
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.8.4
lxml: 3.5.0
bs4: 4.4.1
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: 1.0.11
pymysql: 0.6.2.None
psycopg2: None
Jinja2: 2.8

[troglotit]

It's because of datetime64[ns] overflow. See #6741: other units of datetime64 are not implemented in pandas yet.

Here's dirty workaround:

import pandas as pd
dates = pd.Series(pd.date_range('1700-01-01', periods=4500, freq='m'))
days_delta = (pd.Series(dates.values.astype('uint64') - dates.values.min().astype('uint64')))/86400000000000

[MMCMA]

Thanks for the workaround. Maybe it would make sense to raise at least warning

[TomAugspurger]

Timedeltas and Timestamps are based off numpy's int64, so we follow their wraparound behavior. I believe this is intentional.

In [3]: np.int64(9223372036854775807)
Out[3]: 9223372036854775807

In [4]: np.int64(9223372036854775807) + 1
Out[4]: -9223372036854775808

At the very least we should add this info to the documentation.,

[jreback]

yeh, see the numpy discussion here.

There is kind of a theoretical soln here. we could reduce the magniture when doing ops if they are equally compat

In [19]: (dates.values.astype('i8') == (dates.values.astype('i8')/1000000)*1000000).all()
Out[19]: True

But this would be not be a complete panacea, though might make practical some results.
Keeping in mind that this introduces a perf penalty always