Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MultiIndex can't be indexed with an np.array #15434

Closed
toobaz opened this issue Feb 17, 2017 · 1 comment
Closed

MultiIndex can't be indexed with an np.array #15434

toobaz opened this issue Feb 17, 2017 · 1 comment
Labels
Compat pandas objects compatability with Numpy or Python functions Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex
Milestone

Comments

@toobaz
Copy link
Member

toobaz commented Feb 17, 2017

From #15425

Code Sample, a copy-pastable example if possible

In [3]: mi = pd.MultiIndex.from_product([[1, 2], ['a', 'b']])

In [4]: pd.Series(range(4)).loc[np.array([1, 2])]
Out[4]: 
1    1
2    2
dtype: int64

In [5]: pd.Series(range(4), index=mi).loc[np.array([1, 2])]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in _get_label(self, label, axis)
     99             try:
--> 100                 return self.obj._xs(label, axis=axis)
    101             except:

/home/pietro/nobackup/repo/pandas/pandas/core/generic.py in xs(self, key, axis, level, drop_level)
   1842             loc, new_index = self.index.get_loc_level(key,
-> 1843                                                       drop_level=drop_level)
   1844         else:

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in get_loc_level(self, key, level, drop_level)
   1932                 else:
-> 1933                     return partial_selection(key)
   1934             else:

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in partial_selection(key, indexer)
   1897                     if indexer is None:
-> 1898                         indexer = self.get_loc(key)
   1899                     ilevels = [i for i in range(len(key))

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in get_loc(self, key, method)
   1807         start, stop = (self.slice_locs(lead_key, lead_key)
-> 1808                        if lead_key else (0, len(self)))
   1809 

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in slice_locs(self, start, end, step, kind)
   1713         # happens in get_slice_bound method), but it adds meaningful doc.
-> 1714         return super(MultiIndex, self).slice_locs(start, end, step, kind=kind)
   1715 

/home/pietro/nobackup/repo/pandas/pandas/indexes/base.py in slice_locs(self, start, end, step, kind)
   3346         if start is not None:
-> 3347             start_slice = self.get_slice_bound(start, 'left', kind)
   3348         if start_slice is None:

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in get_slice_bound(self, label, side, kind)
   1684             label = label,
-> 1685         return self._partial_tup_index(label, side=side)
   1686 

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in _partial_tup_index(self, tup, side)
   1727 
-> 1728             if lab not in lev:
   1729                 if not lev.is_type_compatible(lib.infer_dtype([lab])):

/home/pietro/nobackup/repo/pandas/pandas/indexes/base.py in __contains__(self, key)
   1495     def __contains__(self, key):
-> 1496         hash(key)
   1497         # work around some kind of odd cython bug

TypeError: unhashable type: 'numpy.ndarray'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-5-bf8891f2016d> in <module>()
----> 1 pd.Series(range(4), index=mi).loc[np.array([1, 2])]

/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in __getitem__(self, key)
   1339         else:
   1340             key = com._apply_if_callable(key, self.obj)
-> 1341             return self._getitem_axis(key, axis=0)
   1342 
   1343     def _is_scalar_access(self, key):

/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1550         # fall thru to straight lookup
   1551         self._has_valid_type(key, axis)
-> 1552         return self._get_label(key, axis=axis)
   1553 
   1554 

/home/pietro/nobackup/repo/pandas/pandas/core/indexing.py in _get_label(self, label, axis)
    100                 return self.obj._xs(label, axis=axis)
    101             except:
--> 102                 return self.obj[label]
    103         elif isinstance(label, tuple) and isinstance(label[axis], slice):
    104             raise IndexingError('no slices here, handle elsewhere')

/home/pietro/nobackup/repo/pandas/pandas/core/series.py in __getitem__(self, key)
    645             key = check_bool_indexer(self.index, key)
    646 
--> 647         return self._get_with(key)
    648 
    649     def _get_with(self, key):

/home/pietro/nobackup/repo/pandas/pandas/core/series.py in _get_with(self, key)
    658             if isinstance(key, tuple):
    659                 try:
--> 660                     return self._get_values_tuple(key)
    661                 except:
    662                     if len(key) == 1:

/home/pietro/nobackup/repo/pandas/pandas/core/series.py in _get_values_tuple(self, key)
    706 
    707         # If key is contained, would have returned by now
--> 708         indexer, new_index = self.index.get_loc_level(key)
    709         return self._constructor(self._values[indexer],
    710                                  index=new_index).__finalize__(self)

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in get_loc_level(self, key, level, drop_level)
   1931                         return partial_selection(key)
   1932                 else:
-> 1933                     return partial_selection(key)
   1934             else:
   1935                 indexer = None

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in partial_selection(key, indexer)
   1896                 def partial_selection(key, indexer=None):
   1897                     if indexer is None:
-> 1898                         indexer = self.get_loc(key)
   1899                     ilevels = [i for i in range(len(key))
   1900                                if key[i] != slice(None, None)]

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in get_loc(self, key, method)
   1806         lead_key, follow_key = key[:i], key[i:]
   1807         start, stop = (self.slice_locs(lead_key, lead_key)
-> 1808                        if lead_key else (0, len(self)))
   1809 
   1810         if start == stop:

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in slice_locs(self, start, end, step, kind)
   1712         # This function adds nothing to its parent implementation (the magic
   1713         # happens in get_slice_bound method), but it adds meaningful doc.
-> 1714         return super(MultiIndex, self).slice_locs(start, end, step, kind=kind)
   1715 
   1716     def _partial_tup_index(self, tup, side='left'):

/home/pietro/nobackup/repo/pandas/pandas/indexes/base.py in slice_locs(self, start, end, step, kind)
   3345         start_slice = None
   3346         if start is not None:
-> 3347             start_slice = self.get_slice_bound(start, 'left', kind)
   3348         if start_slice is None:
   3349             start_slice = 0

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in get_slice_bound(self, label, side, kind)
   1683         if not isinstance(label, tuple):
   1684             label = label,
-> 1685         return self._partial_tup_index(label, side=side)
   1686 
   1687     def slice_locs(self, start=None, end=None, step=None, kind=None):

/home/pietro/nobackup/repo/pandas/pandas/indexes/multi.py in _partial_tup_index(self, tup, side)
   1726             section = labs[start:end]
   1727 
-> 1728             if lab not in lev:
   1729                 if not lev.is_type_compatible(lib.infer_dtype([lab])):
   1730                     raise TypeError('Level type mismatch: %s' % lab)

/home/pietro/nobackup/repo/pandas/pandas/indexes/base.py in __contains__(self, key)
   1494 
   1495     def __contains__(self, key):
-> 1496         hash(key)
   1497         # work around some kind of odd cython bug
   1498         try:

TypeError: unhashable type: 'numpy.ndarray'

Problem description

The array should be accepted as it is accepted for the (non-Multi)Index

Expected Output

In [6]: pd.Series(range(4), index=mi).loc[np.array([1, 2])]
Out[6]: 
1  a    0
   b    1
2  a    2
   b    3
dtype: int64

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.5.2.final.0 python-bits: 64 OS: Linux OS-release: 4.7.0-1-amd64 machine: x86_64 processor: byteorder: little LC_ALL: None LANG: it_IT.utf8 LOCALE: it_IT.UTF-8

pandas: 0.19.0+466.g5a8883b
pytest: 3.0.6
pip: 8.1.2
setuptools: 28.0.0
Cython: 0.23.4
numpy: 1.12.0
scipy: 0.18.1
xarray: None
IPython: 5.1.0.dev
sphinx: 1.4.8
patsy: 0.3.0-dev
dateutil: 2.5.3
pytz: 2015.7
blosc: None
bottleneck: 1.2.0
tables: 3.2.2
numexpr: 2.6.0
feather: None
matplotlib: 2.0.0rc2
openpyxl: 2.3.0
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: 0.999
httplib2: 0.9.1
apiclient: 1.5.2
sqlalchemy: 1.0.15
pymysql: None
psycopg2: None
jinja2: 2.8
s3fs: None
pandas_datareader: 0.2.1

@toobaz toobaz mentioned this issue Feb 17, 2017
5 tasks
@jreback
Copy link
Contributor

jreback commented Feb 17, 2017

why don't you add the fix for this in #15425 as well (as its quite similar). Be sure that the ndarray is 1d though (if its NOT 1d it should raise), and pls test this.

@jreback jreback added Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex Compat pandas objects compatability with Numpy or Python functions and removed Bug labels Feb 17, 2017
@jreback jreback added this to the Next Major Release milestone Feb 17, 2017
toobaz added a commit to toobaz/pandas that referenced this issue Feb 17, 2017
@jreback jreback modified the milestones: 0.20.0, Next Major Release Feb 18, 2017
AnkurDedania pushed a commit to AnkurDedania/pandas that referenced this issue Mar 21, 2017
…xers

closes pandas-dev#15424
closes pandas-dev#15434

Author: Pietro Battiston <[email protected]>

Closes pandas-dev#15425 from toobaz/mi_indexing and squashes the following commits:

2ba2d5d [Pietro Battiston] Updated comment
900e3ce [Pietro Battiston] whatsnew
8467b57 [Pietro Battiston] Tests for previous commit
17209f3 [Pietro Battiston] BUG: support indexing MultiIndex with 1-D array
7606114 [Pietro Battiston] Whatsnew
0b719f5 [Pietro Battiston] Test for previous commit
1f2f385 [Pietro Battiston] BUG: Fix indexing MultiIndex with Series with 0 not index
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants