-
-
Notifications
You must be signed in to change notification settings - Fork 18.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: InvalidIndexError raised in Categorical.isin for categorical backed by interval with overlapping intervals #34974
Labels
Comments
TomAugspurger
added
Bug
Categorical
Categorical Data Type
Interval
Interval data type
labels
Jun 24, 2020
I have the same issue (I think) with just straight up J
Out[9]:
s r c
live_range
0 [0, 3] 1024 0 3
1 [1, 3] 1024 1 3
2 [3, 4] 1024 3 4
3 [5, 10] 256 5 10
4 [6, 14] 256 6 14
5 [7, 10] 256 7 10
6 [8, 9] 256 8 9
7 [9, 12] 256 9 12
8 [10, 12] 256 10 12
9 [13, 14] 256 13 14
j.Index
Out[10]: (0, Interval(0, 3, closed='both'))
J.drop(j.Index)
Traceback (most recent call last):
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3441, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-11-ad3fd2b03dcb>", line 1, in <module>
J.drop(j.Index)
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/frame.py", line 4901, in drop
return super().drop(
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/generic.py", line 4147, in drop
obj = obj._drop_axis(labels, axis, level=level, errors=errors)
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/generic.py", line 4183, in _drop_axis
result = self.reindex(**{axis_name: new_axis})
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/util/_decorators.py", line 324, in wrapper
return func(*args, **kwargs)
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/frame.py", line 4767, in reindex
return super().reindex(**kwargs)
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/generic.py", line 4809, in reindex
return self._reindex_axes(
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/frame.py", line 4592, in _reindex_axes
frame = frame._reindex_index(
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/frame.py", line 4608, in _reindex_index
new_index, indexer = self.index.reindex(
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 2520, in reindex
indexer = self.get_indexer(
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3486, in get_indexer
return self._get_indexer(target, method, limit, tolerance)
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 2682, in _get_indexer
indexer = self._engine.get_indexer(target._values)
File "pandas/_libs/index.pyx", line 623, in pandas._libs.index.BaseMultiIndexCodesEngine.get_indexer
File "pandas/_libs/index.pyx", line 603, in pandas._libs.index.BaseMultiIndexCodesEngine._extract_level_codes
File "/Users/mlevental/opt/miniconda3/envs/pytorch_dev_shape_inference/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3442, in get_indexer
raise InvalidIndexError(self._requires_unique_msg)
pandas.errors.InvalidIndexError: cannot handle overlapping indices; use IntervalIndex.get_indexer_non_unique what's the solution here? EDIT: some kind of class hierarchy hijinks: self._index_as_unique
Out[19]: False
self.is_unique
Out[20]: True EDIT2: if anyone gets here and wants a hack fix (as of 1.3.1): drop def _validate_can_reindex(self, indexer: np.ndarray) -> None:
if isinstance(self, IntervalIndex):
is_unique = self.is_unique
else:
is_unique = self._index_as_unique
if not is_unique and len(indexer):
raise ValueError("cannot reindex from a duplicate axis") |
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Code Sample, a copy-pastable example
Problem description
The result is unambigous
Expected Output
I think that the call to
self.categories.get_indexer
might beself.categories.get_indexer_for
instead.The text was updated successfully, but these errors were encountered: