Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pandas 1.0.1 fails get_loc on timeindex (0.25.3 works) #32140

Closed
AndrewMoscoe opened this issue Feb 21, 2020 · 7 comments · Fixed by #32152
Closed

Pandas 1.0.1 fails get_loc on timeindex (0.25.3 works) #32140

AndrewMoscoe opened this issue Feb 21, 2020 · 7 comments · Fixed by #32152
Labels
Needs Tests Unit test(s) needed to prevent regressions
Milestone

Comments

@AndrewMoscoe
Copy link

findtime = pd.Timestamp('2019-12-12 10:19:25', tz='US/Eastern')
start = pd.Timestamp('2019-12-12 0:0:0', tz='US/Eastern')
end = pd.Timestamp('2019-12-13 0:0:0', tz='US/Eastern')
testindex = pd.date_range(start, end, freq='5s')
testindex.get_loc(findtime, method='nearest')

Problem description

With pandas 1.0.1, python 3.8.1

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python38\lib\site-packages\pandas\core\indexes\datetimes.py", line 699, in get_loc
    return Index.get_loc(self, key, method, tolerance)
  File "C:\Program Files\Python38\lib\site-packages\pandas\core\indexes\base.py", line 2649, in get_loc
    indexer = self.get_indexer([key], method=method, tolerance=tolerance)
  File "C:\Program Files\Python38\lib\site-packages\pandas\core\indexes\base.py", line 2740, in get_indexer
    indexer = self._get_nearest_indexer(target, limit, tolerance)
  File "C:\Program Files\Python38\lib\site-packages\pandas\core\indexes\base.py", line 2821, in _get_nearest_indexer
    left_distances = abs(self.values[left_indexer] - target)
numpy.core._exceptions.UFuncTypeError: ufunc 'subtract' cannot use operands with types dtype('<M8[ns]') and dtype('O')

With 0.25.3, this returns index location 7433 as expected.

I've tested this on Windows 10 1909, and Ubuntu WSL, and get the same error on both.

Output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.8.1.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
machine          : AMD64
processor        : AMD64 Family 23 Model 8 Stepping 2, AuthenticAMD
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : English_Canada.1252

pandas           : 1.0.1
numpy            : 1.18.1
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 20.0.2
setuptools       : 41.2.0
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : 4.5.0
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : 2.11.1
IPython          : 7.12.0
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : 4.5.0
matplotlib       : 3.1.3
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : None
pyxlsb           : None
s3fs             : None
scipy            : 1.4.1
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
numba            : None
@jorisvandenbossche
Copy link
Member

@AndrewMoscoe Thanks for the report!

This is already working again on master. But so we should verify it if also works on the 1.0.x branch (to become 1.0.2), or if there is something we need to backport.

@jorisvandenbossche jorisvandenbossche added the Needs Tests Unit test(s) needed to prevent regressions label Feb 21, 2020
@jorisvandenbossche jorisvandenbossche added this to the 1.0.2 milestone Feb 21, 2020
@simonjayhawkins
Copy link
Member

@jorisvandenbossche This bug was noticed in #26683 (comment) and fixed in #31511.

so we need to backport that PR. and move the whatsnew. any particular order? Also, do we need to add another test for this?

@jorisvandenbossche
Copy link
Member

The order is not that important. I think the easiest is to backport the PR (can first try asking MeeseeksDev, in case there is no conflict), and update the branch with moving the whatsnew. And then a separate PR moving the whatsnew just in master.

I would also still add a test, since the case seems a bit different.

@simonjayhawkins
Copy link
Member

xref #31964

@simonjayhawkins
Copy link
Member

OK will

  1. backport BUG: fix reindexing with a tz-aware index and method='nearest' #31511 with whatsnew moved.
  2. PR to mater with whatsnew move
  3. PR to master with this code sample as test case (no need to backport?) to close this issue
  4. PR to master with code sample from Reindex on nearest datetime gives UFuncTypeError - subtract with dtype('<M8[ns]') and dtype('O') #31964 as test case to close that issue

@jorisvandenbossche
Copy link
Member

Sounds good, thanks!

@simonjayhawkins
Copy link
Member

(can first try asking MeeseeksDev, in case there is no conflict)

had already cherry-picked locally to test. so just created backport manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants