Comparing Strings to Numbers: No ValueError

[mattayes]

I noticed this quirk today: When you do a DataFrame-wide comparison (excluding ==) using a number, it doesn't raise a ValueError (which you'd expect in Python 3); instead it always returns True.

>>> from pandas import DataFrame
>>> df = DataFrame(x: {'x': 'foo', 'y': 'bar', 'z': 'baz'} for x in ['a', 'b', 'c']})
>>> df
     a    b    c
x  foo  foo  foo
y  bar  bar  bar
z  baz  baz  baz
>>> df < 0
      a     b     c
x  True  True  True
y  True  True  True
z  True  True  True
>>> df > 0
      a     b     c
x  True  True  True
y  True  True  True
z  True  True  True

However, when you compare a Series of strings to a number, you get the expected ValueError:

>>> df.a < 0
TypeError: unorderable types: str() < int()

Is this a bug or a feature?

Python: 3.4.3
Pandas: 0.17.0
OS: Mac OSX 10.11

[jreback]

yeh, I think this is an api inconsistency. we should make this work I think for df.a < 0 as it works for everything else, and raising an error is not useful here.

Note that we do have code to do this comparision even in py3, see core/algorithms.py/factorize so I think that could be incorporated into the comparison routines (in ops.py)

[mattayes]

Oh, really? I think the error is helpful for Series. What I find problematic is that strings always return True on a DataFrame-wide comparison. I'd expect None or at the very least False.

What I originally wanted to do was replace all negative values in the DataFrame with np.nan and then df.dropna(). Under this structure I can't, because the strings will be replaced as well, leaving me with a rather empty DataFrame. Of course, if I wanted to replace all the positive values `I still couldn't!

Thoughts?

[mroeschke]

This looks to raise correctly on master. Could use a test.

In [68]: df < 0
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: '>' not supported between instances of 'str' and 'int'

In [71]: pd.__version__
Out[71]: '0.26.0.dev0+533.gd8f9be7e3'