Mi indexing #15425

toobaz · 2017-02-16T15:44:48Z

closes Indexing MultiIndex with NDFrame indexer fails if index of indexer does not contain 0 #15424
closes MultiIndex can't be indexed with an np.array #15434
tests added / passed
passes git diff upstream/master | flake8 --diff
whatsnew entry

This also fixes indexing with a DataFrame, which was actually not mentioned in #15424 (but it's a twoliner)

jreback · 2017-02-16T17:50:38Z

pandas/tests/indexing/test_multiindex.py

+        # passing a dataframe as a key with a MultiIndex
+        index = MultiIndex.from_product([[1, 2, 3], ['A', 'B', 'C']])
+        x = Series(index=index, data=range(9), dtype=np.float64)
+        idx_keys = [(1, 'A'), (2, 'C'), (3, 'B')]


can you also add tests for empty Series/DataFrame indexers (Series in test above)

Lost again... aren't you referring to something like this and this?!

use an empty series / dataframe as an indexer (no data)

Series(data=[], dtype=np.float64) isn't empty?!

ahh it's in the above be test ok then

jreback · 2017-02-16T17:51:03Z

lgtm. just a small test addition. ping when pushed and green.

codecov-io · 2017-02-16T23:04:45Z

Codecov Report

Merging #15425 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #15425      +/-   ##
==========================================
+ Coverage   90.37%   90.37%   +<.01%     
==========================================
  Files         135      135              
  Lines       49463    49467       +4     
==========================================
+ Hits        44702    44707       +5     
+ Misses       4761     4760       -1

Impacted Files	Coverage Δ
pandas/core/indexing.py	`94.22% <100%> (+0.02%)`	✅
pandas/core/common.py	`91.36% <ø> (+0.33%)`	✅

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f65a641...2ba2d5d. Read the comment docs.

jorisvandenbossche · 2017-02-16T23:39:44Z

The ability to index with a DataFrame is actually a new feature, not a bug fix. And thus it should also documented like that if we include it.
But, personally, I would not add this. I don't see a compelling reason to have this, and it will only add further complexity to indexing (eg, should the column names align on the level names?)

toobaz · 2017-02-17T00:09:45Z

Ouch, right. I caught #15424 while trying to index with a DataFrame and getting a KeyError: 0, this led me to think it was a bug and not a missing feature, sorry.

That said, I would love that feature. I understand the "complexity" argument, but I propose to reason backwards: if we don't want the column names to align on the level names, then there's no complexity added - it's already done (and really, DataFrames become tuples so early that you don't have to special-case anything else). If instead we want this to happen, I will see what I can do.

Concerning this PR: in the first case, I would just add a whatsnew section, while in the second, I would drop the two incriminated lines (and the related test).

toobaz · 2017-02-17T00:18:45Z

By the way: indexing with a Series is also not documented, right?

~~(only now I realize that not even .loc[np.array] is supported... which instead we might want to I guess)~~ moved to #15434

jreback · 2017-02-17T14:55:11Z

yeah I agree with @jorisvandenbossche how is this actually useful? it is just plain confusing, e.g. what dimensions are you actually indexing on? what happens if the levels don't match, or partially match.

I think this is too much of a rabbit hole. Take this our for now (and just leave the series fixing).

Ok with actually raising NotImplemented Error though if a df is passed like this.

You can raise a separate issue with some well-defined use cases for consideration though.

closes pandas-dev#15434

jreback · 2017-02-17T18:08:54Z

pandas/core/indexing.py

+                    raise NotImplementedError("Indexing a MultiIndex with a "
+                                              "DataFrame key is not "
+                                              "implemented")
+                elif hasattr(key, 'ndim') and key.ndim > 1:


test using a numpy scalar as well (which is 0-dim), e.g. np.int64(5) (I think this works anyhow)

jreback · 2017-02-17T18:09:19Z

pandas/core/indexing.py

@@ -1525,11 +1525,22 @@ def _getitem_axis(self, key, axis=0):
            # possibly convert a list-like into a nested tuple
            # but don't convert a list-like of tuples


can you update this a bit (explain the goal of what is happening)

toobaz · 2017-02-17T23:13:58Z

pandas/core/indexing.py

@@ -1521,15 +1521,24 @@ def _getitem_axis(self, key, axis=0):
            return self._getbool_axis(key, axis=axis)
        elif is_list_like_indexer(key):

-            # GH 7349
-            # possibly convert a list-like into a nested tuple
-            # but don't convert a list-like of tuples


Related to (the code I currently don't understand in) #15448

jreback · 2017-02-18T00:37:22Z

lgtm. ping on green.

jreback · 2017-02-20T14:43:57Z

thanks @toobaz

…xers closes pandas-dev#15424 closes pandas-dev#15434 Author: Pietro Battiston <[email protected]> Closes pandas-dev#15425 from toobaz/mi_indexing and squashes the following commits: 2ba2d5d [Pietro Battiston] Updated comment 900e3ce [Pietro Battiston] whatsnew 8467b57 [Pietro Battiston] Tests for previous commit 17209f3 [Pietro Battiston] BUG: support indexing MultiIndex with 1-D array 7606114 [Pietro Battiston] Whatsnew 0b719f5 [Pietro Battiston] Test for previous commit 1f2f385 [Pietro Battiston] BUG: Fix indexing MultiIndex with Series with 0 not index

jreback added Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex labels Feb 16, 2017

toobaz mentioned this pull request Feb 16, 2017

Indexing MultiIndex with NDFrame indexer fails if index of indexer does not contain 0 #15424

Closed

jreback reviewed Feb 16, 2017

View reviewed changes

toobaz mentioned this pull request Feb 17, 2017

MultiIndex can't be indexed with an np.array #15434

Closed

toobaz mentioned this pull request Feb 17, 2017

Adding support for indexing a MultiIndex with a DataFrame and/or bi-dimensional np.array #15438

Open

toobaz force-pushed the mi_indexing branch from c516c45 to c6c52b0 Compare February 17, 2017 16:20

toobaz added 3 commits February 17, 2017 17:47

BUG: Fix indexing MultiIndex with Series with 0 not index

1f2f385

Test for previous commit

0b719f5

Whatsnew

7606114

toobaz force-pushed the mi_indexing branch from c6c52b0 to 7606114 Compare February 17, 2017 16:48

BUG: support indexing MultiIndex with 1-D array

17209f3

closes pandas-dev#15434

toobaz force-pushed the mi_indexing branch from 7ab5736 to 5050d6b Compare February 17, 2017 17:38

jreback reviewed Feb 17, 2017

View reviewed changes

toobaz force-pushed the mi_indexing branch from 5050d6b to 44f00b6 Compare February 17, 2017 23:07

toobaz added 3 commits February 18, 2017 00:10

Tests for previous commit

8467b57

whatsnew

900e3ce

Updated comment

2ba2d5d

toobaz force-pushed the mi_indexing branch from 44f00b6 to 2ba2d5d Compare February 17, 2017 23:11

toobaz commented Feb 17, 2017

View reviewed changes

jreback added this to the 0.20.0 milestone Feb 18, 2017

jreback closed this in 821be39 Feb 20, 2017

toobaz deleted the mi_indexing branch February 21, 2017 01:00

toobaz mentioned this pull request Feb 22, 2017

Indexing a MultiIndex with a (Multi)Index #15472

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mi indexing #15425

Mi indexing #15425

toobaz commented Feb 16, 2017 •

edited

Loading

jreback Feb 16, 2017

toobaz Feb 16, 2017

jreback Feb 16, 2017

toobaz Feb 16, 2017

jreback Feb 16, 2017

jreback commented Feb 16, 2017

codecov-io commented Feb 16, 2017 •

edited

Loading

jorisvandenbossche commented Feb 16, 2017

toobaz commented Feb 17, 2017

toobaz commented Feb 17, 2017 •

edited

Loading

jreback commented Feb 17, 2017

jreback Feb 17, 2017

toobaz Feb 17, 2017

jreback Feb 17, 2017

toobaz Feb 17, 2017

jreback commented Feb 18, 2017

jreback commented Feb 20, 2017

		@@ -1525,11 +1525,22 @@ def _getitem_axis(self, key, axis=0):
		# possibly convert a list-like into a nested tuple
		# but don't convert a list-like of tuples

Mi indexing #15425

Mi indexing #15425

Conversation

toobaz commented Feb 16, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Feb 16, 2017

codecov-io commented Feb 16, 2017 • edited Loading

Codecov Report

jorisvandenbossche commented Feb 16, 2017

toobaz commented Feb 17, 2017

toobaz commented Feb 17, 2017 • edited Loading

jreback commented Feb 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Feb 18, 2017

jreback commented Feb 20, 2017

toobaz commented Feb 16, 2017 •

edited

Loading

codecov-io commented Feb 16, 2017 •

edited

Loading

toobaz commented Feb 17, 2017 •

edited

Loading