REGR: Array.__setitem__ failing with nullable boolean mask

[mcitoler]

Code Sample, a copy-pastable example if possible

import pandas as pd
s = pd.Series([1, 2], dtype='Int64') 
s.where(s > 1)

Problem description

The code above raises

TypeError: bad operand type for unary ~: 'BooleanArray'

Expected Output

0    <NA>
1       2
dtype: Int64

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.8.1.final.0
python-bits : 64
OS : Linux
OS-release : 4.15.0-76-generic
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.0.0rc0
numpy : 1.18.1
pytz : 2019.3
dateutil : 2.8.1
pip : 20.0.2
setuptools : 45.1.0.post20200119
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.0
IPython : 7.11.1
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : None

[MarcoGorelli]

Thanks for the report

The code above raises

TypeError: bad operand type for unary ~: 'BooleanArray'

When I tried this on master I got

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

[jorisvandenbossche]

Thanks, this is a regression (due to the masked array / boolean NA changes) compared to 0.25.3, so tagged it with the 1.0.1 milestone.

We recently did some work to check and sanitize indexers in __getitem__ (including converting BooleanArray to numpy boolean arrays (it's not having this numpy array that is causing the error here), see #31150 and #30308), but we will need to do something similar for __setitem__

[jorisvandenbossche]

As example, also a direct setitem on the array (and not through Series.where) fails as well:

In [9]: import pandas as pd 
   ...: s = pd.Series([1, 2], dtype='Int64') 

In [10]: arr = s.array   

In [11]: arr[arr > 1] = 1  
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-11-30a6ee9cc6a5> in <module>
----> 1 arr[arr > 1] = 1

~/scipy/pandas/pandas/core/arrays/integer.py in __setitem__(self, key, value)
    415             mask = mask[0]
    416 
--> 417         self._data[key] = value
    418         self._mask[key] = mask
    419 

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices