BUG: np.all cannot handle a boolean pd.Index anymore

[jolespin]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd,  numpy as np
print(pd.__version__)
print(np.__version__)
# 1.2.2
# 1.19.5

conditions = pd.Index([True, True, True, True, True, True], dtype='object')
conditions
# Index([True, True, True, True, True, True], dtype='object')
np.all(conditions.values)
# True
np.all(conditions)
# ---------------------------------------------------------------------------
# TypeError                                 Traceback (most recent call last)
# <ipython-input-102-9a6b94382bda> in <module>
# ----> 1 np.all(conditions)

# <__array_function__ internals> in all(*args, **kwargs)

# ~/anaconda3/envs/soothsayer5_env/lib/python3.8/site-packages/numpy/core/fromnumeric.py in all(a, axis, out, keepdims)
#    2409 
#    2410     """
# -> 2411     return _wrapreduction(a, np.logical_and, 'all', axis, None, out, keepdims=keepdims)
#    2412 
#    2413 

# ~/anaconda3/envs/soothsayer5_env/lib/python3.8/site-packages/numpy/core/fromnumeric.py in _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs)
#      83                 return reduction(axis=axis, dtype=dtype, out=out, **passkwargs)
#      84             else:
# ---> 85                 return reduction(axis=axis, out=out, **passkwargs)
#      86 
#      87     return ufunc.reduce(obj, axis, dtype, out, **passkwargs)

# TypeError: all() got an unexpected keyword argument 'axis'

Problem description

[this should explain why the current behaviour is a problem and why the expected output is a better solution]
For years I've been using pandas index arrays with only True/False entries. np.all was always able to handle these and for some reason it cannot anymore. I'm not sure why and the error message is not intuitive to me since I'm not using an axis keyword.

Expected Output

True or False

Output of pd.show_versions()

[paste the output of pd.show_versions() here leaving a blank line after the details tag]

INSTALLED VERSIONS

commit : 7d32926
python : 3.8.5.final.0
python-bits : 64
OS : Darwin
OS-release : 19.5.0
Version : Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.2
numpy : 1.19.5
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.2
setuptools : 49.3.1.post20200810
Cython : 0.29.21
pytest : 6.0.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : 1.3.7
lxml.etree : 4.5.2
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.17.0
pandas_datareader: None
bs4 : 4.9.3
bottleneck : None
fsspec : 0.8.0
fastparquet : 0.4.0
gcsfs : None
matplotlib : 3.3.1
numexpr : None
odfpy : None
openpyxl : 3.0.3
pandas_gbq : None
pyarrow : 1.0.1
pyxlsb : None
s3fs : None
scipy : 1.5.2
sqlalchemy : 1.3.16
tables : None
tabulate : None
xarray : 0.16.2
xlrd : 1.2.0
xlwt : None
numba : 0.50.1

[jolespin]

I can confirm that this happens in v1.2.3:

>>> import numpy as np
>>> import pandas as pd
>>> pd.__version__
'1.2.3'
>>> conditions = pd.Index([True, True, True, True, True, True], dtype='object')
>>> np.all(conditions)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<__array_function__ internals>", line 5, in all
  File "/Users/jespinoz/anaconda3/envs/testing_env/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 2438, in all
    return _wrapreduction(a, np.logical_and, 'all', axis, None, out,
  File "/Users/jespinoz/anaconda3/envs/testing_env/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 85, in _wrapreduction
    return reduction(axis=axis, out=out, **passkwargs)
TypeError: all() got an unexpected keyword argument 'axis'

and does work in v1.0.2:

>>> import numpy as np
>>> import pandas as pd
>>> pd.__version__
'1.0.2'
>>> conditions = pd.Index([True, True, True, True, True, True], dtype='object')
>>> np.all(conditions)
True

[lithomas1]

I cannot reproduce this with numpy versions 1.17.2 or 1.18.0 with master. Can you try downgrading your numpy version to see if that works?

EDIT: Tried also on numpy 1.19.5 on master and it works.

[jolespin]

When I changed my pandas version above, it changed my numpy version to v1.20.1.

I tried your suggestion above:

>>> import numpy as np
>>> np.__version__
'1.18.0'
>>> import pandas as pd
^[[Apd.__version__
'1.2.3'
>>> conditions = pd.Index([True, True, True, True, True, True], dtype='object')
>>> np.all(conditions)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<__array_function__ internals>", line 5, in all
  File "/Users/jespinoz/anaconda3/envs/testing_env/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 2398, in all
    return _wrapreduction(a, np.logical_and, 'all', axis, None, out, keepdims=keepdims)
  File "/Users/jespinoz/anaconda3/envs/testing_env/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 88, in _wrapreduction
    return reduction(axis=axis, out=out, **passkwargs)
TypeError: all() got an unexpected keyword argument 'axis'

[lithomas1]

@jolespin This seems to be a regression starting from 1.2.0 as I cannot reproduce on 1.1.5 and master. Can you try again with the master branch on pandas(requires building from source) to verify that it works?

[jolespin]

Looks like it's working again with the current pandas in development.

What changed between v1.2.3 and v1.3 that could cause this?

>>> import numpy as np
>>> np.__version__
'1.18.0'
>>> import pandas as pd
>>> pd.__version__
'1.3.0.dev0+970.g154026cc71'
>>> conditions = pd.Index([True, True, True, True, True, True], dtype='object')
>>> np.all(conditions)
True

Works with updated numpy as well:

>>> import numpy as np
>>> np.__version__
'1.20.1'
>>> import pandas as pd
pd.__version__
'1.3.0.dev0+970.g154026cc71'
>>> conditions = pd.Index([True, True, True, True, True, True], dtype='object')
>>> np.all(conditions)
True

[MarcoGorelli]

What changed between v1.2.3 and v1.3 that could cause this?

Thanks for the report @jolespin - looks like it was fixed intentionally in #40180

539b8134194bf41526d2dc9e1b194c6e54eb683e is the first bad commit
commit 539b8134194bf41526d2dc9e1b194c6e54eb683e
Date:   Fri Mar 5 01:28:49 2021 +0000

    Make Index.all compatible with numpy.all (#40180)

 doc/source/whatsnew/v1.3.0.rst             |  2 ++
 pandas/compat/numpy/function.py            |  1 +
 pandas/core/indexes/base.py                | 15 +++++++--------
 pandas/tests/reductions/test_reductions.py |  9 +++++++++
 4 files changed, 19 insertions(+), 8 deletions(-)
bisect run success

so I think we can close

[jolespin]

I’ll close the corresponding numpy issue. Thanks!

Is this available in any stable version?