Skip to content

Commit

Permalink
DOC: enhanced docs for #10569
Browse files Browse the repository at this point in the history
  • Loading branch information
jreback committed Jul 17, 2015
1 parent 0de48d0 commit 2da060c
Show file tree
Hide file tree
Showing 2 changed files with 61 additions and 8 deletions.
28 changes: 21 additions & 7 deletions doc/source/missing_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,26 +68,40 @@ detect this value with data of different types: floating point, integer,
boolean, and general object. In many cases, however, the Python ``None`` will
arise and we wish to also consider that "missing" or "null".

Prior to version v0.10.0 ``inf`` and ``-inf`` were also
considered to be "null" in computations. This is no longer the case by
default; use the ``mode.use_inf_as_null`` option to recover it.
.. note::

Prior to version v0.10.0 ``inf`` and ``-inf`` were also
considered to be "null" in computations. This is no longer the case by
default; use the ``mode.use_inf_as_null`` option to recover it.

.. _missing.isnull:

To make detecting missing values easier (and across different array dtypes),
pandas provides the :func:`~pandas.core.common.isnull` and
:func:`~pandas.core.common.notnull` functions, which are also methods on
``Series`` objects:
``Series`` and ``DataFrame`` objects:

.. ipython:: python
df2['one']
pd.isnull(df2['one'])
df2['four'].notnull()
df2.isnull()
.. warning::

One has to be mindful that in python (and numpy), the ``nan's`` don't compare equal, but ``None's`` **do**.

.. ipython:: python
None == None
np.nan == np.nan
So as compared to above, a scalar equality comparison versus a ``None/np.nan`` doesn't provide useful information.

.. ipython:: python
**Summary:** ``NaN`` and ``None`` (in object arrays) are considered
missing by the ``isnull`` and ``notnull`` functions. ``inf`` and
``-inf`` are no longer considered missing by default.
df2['one'] == np.nan
Datetimes
---------
Expand Down
41 changes: 40 additions & 1 deletion doc/source/whatsnew/v0.17.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,46 @@ or it can return False if broadcasting can not be done:

np.array([1, 2, 3]) == np.array([1, 2])

Changes to Boolean Comparisons vs. None
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Boolean comparisons of a ``Series`` vs ``None`` will now be equivalent to comparing with ``np.nan``, rather than raise ``TypeError``. xref (:issue:`1079`).

.. ipython:: python

s = Series(range(3.))
s.iloc[1] = None
s

Previous behavior:

.. code-block:: python

In [5]: s==None
TypeError: Could not compare <type 'NoneType'> type with Series

New behavior:

.. ipython:: python

s==None

Usually you simply want to know which values are null.

.. ipython:: python

s.isnull()

.. warning::

You generally will want to use ``isnull/notnull`` for these types of comparisons, as ``isnull/notnull`` tells you which elements are null. One has to be
mindful that ``nan's`` don't compare equal, but ``None's`` do.

.. ipython:: python

None == None
np.nan == np.nan

Other API Changes
^^^^^^^^^^^^^^^^^

Expand Down Expand Up @@ -285,7 +325,6 @@ Performance Improvements
Bug Fixes
~~~~~~~~~

- Boolean comparisons of a ``Series`` vs ``None`` will now be equivalent to comparing with ``np.nan``, rather than raise ``TypeError``, xref (:issue:`1079`).
- Bug in ``DataFrame.apply`` when function returns categorical series. (:issue:`9573`)
- Bug in ``to_datetime`` with invalid dates and formats supplied (:issue:`10154`)
- Bug in ``Index.drop_duplicates`` dropping name(s) (:issue:`10115`)
Expand Down

0 comments on commit 2da060c

Please sign in to comment.