Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API/BUG: pd.NA == pd.NaT returns False #34104

Closed
jbrockmendel opened this issue May 10, 2020 · 2 comments · Fixed by #50901
Closed

API/BUG: pd.NA == pd.NaT returns False #34104

jbrockmendel opened this issue May 10, 2020 · 2 comments · Fixed by #50901
Labels
Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments

@jbrockmendel
Copy link
Member

@jorisvandenbossche I expected this to return pd.NA. Is this intentional?

@jbrockmendel jbrockmendel added Bug Needs Triage Issue that has not been reviewed by a pandas team member API Design and removed Needs Triage Issue that has not been reviewed by a pandas team member labels May 10, 2020
@jorisvandenbossche
Copy link
Member

The same is true for any datetime-like scalar, not just NaT:

In [9]: pd.NA == pd.Timestamp("2012")
Out[9]: False

I don't think this is intentional, but somewhat "knowingly ignored for now", at least going from this comment: #30245 (comment) (although that specific comment was about ufuncs, but for scalar operations it's the same logic).

Right now, we only handle a few specific scalar types (and for other scalars, it defers by returning NotImplemented):

def _create_binary_propagating_op(name, is_divmod=False):
def method(self, other):
if (other is C_NA or isinstance(other, str)
or isinstance(other, (numbers.Number, np.bool_))
or isinstance(other, np.ndarray) and not other.shape):
# Need the other.shape clause to handle NumPy scalars,
# since we do a setitem on `out` below, which
# won't work for NumPy scalars.
if is_divmod:
return NA, NA
else:
return NA

So either we need to handle more scalar types here, or our scalar types need to handle NA.

@jbrockmendel
Copy link
Member Author

i think the relevant Timestamp/NaT methods have been updated so they will return NotImplemented when operating against pd.NA, so it would now be easy to change the behavior if desired

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants