CFTimeIndex

doc/time-series.rst

@@ -70,7 +70,10 @@ You can manual decode arrays in this form by passing a dataset to
 One unfortunate limitation of using ``datetime64[ns]`` is that it limits the
 native representation of dates to those that fall between the years 1678 and
 2262. When a netCDF file contains dates outside of these bounds, dates will be
-returned as arrays of ``netcdftime.datetime`` objects.
+returned as arrays of ``netcdftime.datetime`` objects and a ``NetCDFTimeIndex``
+can be used for indexing.  The ``NetCDFTimeIndex`` enables only a subset of
+the indexing functionality of a ``pandas.DatetimeIndex``.  See
+:ref:`NetCDFTimeIndex` for more information.
 
 Datetime indexing
 -----------------
@@ -207,3 +210,90 @@ Dataset and DataArray objects with an arbitrary number of dimensions.
 
 For more examples of using grouped operations on a time dimension, see
 :ref:`toy weather data`.
+
+
+.. _NetCDFTimeIndex:
+
+Non-standard calendars and dates outside the Timestamp-valid range
+------------------------------------------------------------------
+
+Through the optional ``netcdftime`` library and a custom subclass of
+``pandas.Index``, xarray supports a subset of the indexing functionality enabled
+through the standard ``pandas.DatetimeIndex`` for dates from non-standard
+calendars or dates using a standard calendar, but outside the
+`Timestamp-valid range`_ (approximately between years 1678 and 2262).  This
+behavior has not yet been turned on by default; to take advantage of this
+functionality, you must have the ``enable_netcdftimeindex`` option set to
+``True`` within your context (see :py:func:`~xarray.set_options` for more
+information).
+
+For instance, you can create a DataArray indexed by a time
+coordinate with a no-leap calendar within a context manager setting the
+``enable_netcdftimeindex`` option, and the time index will be cast to a
+``NetCDFTimeIndex``:
+
+.. ipython:: python
+
+   from itertools import product
+   from netcdftime import DatetimeNoLeap
+
+   dates = [DatetimeNoLeap(year, month, 1) for year, month in
+            product(range(1, 3), range(1, 13))]
+   with xr.set_options(enable_netcdftimeindex=True):
+       da = xr.DataArray(np.arange(24), coords=[dates], dims=['time'],
+                         name='foo')
+
+.. note::
+
+   With the ``enable_netcdftimeindex`` option activated, a ``NetCDFTimeIndex``
+   will be used for time indexing if any of the following are true:
+
+   - The dates are from a non-standard calendar
+   - Any dates are outside the Timestamp-valid range
+
+   Otherwise a ``pandas.DatetimeIndex`` will be used.  In addition, if any
+   variable (not just an index variable) is encoded using a non-standard
+   calendar, its times will be decoded into ``netcdftime.datetime`` objects,
+   regardless of whether or not they can be represented using
+   ``np.datetime64[ns]`` objects.
+
+For data indexed by a ``NetCDFTimeIndex`` xarray currently supports `partial
+datetime string indexing`_ using strictly `ISO 8601-format`_ partial datetime
+strings:
+
+.. ipython:: python
+
+   da.sel(time='0001')
+   da.sel(time=slice('0001-05', '0002-02'))
+
+access of basic datetime components via the ``dt`` accessor (in this case just
+"year", "month", "day", "hour", "minute", "second", "microsecond", and "season"):
+
+.. ipython:: python
+
+   da.time.dt.year
+   da.time.dt.month
+   da.time.dt.season
+
+group-by operations based on datetime accessor attributes (e.g. by month of the
+year):
+
+.. ipython:: python
+
+   da.groupby('time.month').sum()
+
+and serialization:
+
+.. ipython:: python
+
+   da.to_netcdf('example.nc')
+   xr.open_dataset('example.nc')
+
+.. note::
+
+   Currently resampling along the time dimension for data indexed by a
+   ``NetCDFTimeIndex`` is not supported.
+
+.. _Timestamp-valid range: https://pandas.pydata.org/pandas-docs/stable/timeseries.html#timestamp-limitations
+.. _ISO 8601-format: https://en.wikipedia.org/wiki/ISO_8601
+.. _partial datetime string indexing: https://pandas.pydata.org/pandas-docs/stable/timeseries.html#partial-string-indexing

doc/whats-new.rst

@@ -37,6 +37,15 @@ Documentation
 Enhancements
 ~~~~~~~~~~~~
 
+- Add an option for using a ``NetCDFTimeIndex`` for indexing times with
+  non-standard calendars and/or outside the Timestamp-valid range; this index
+  enables a subset of the functionality of a standard
+  ``pandas.DatetimeIndex`` (:issue:`789`, :issue:`1084`, :issue:`1252`).
+  By `Spencer Clark <https://github.com/spencerkclark>`_ with help from
+  `Stephan Hoyer <https://github.com/shoyer>`_.
+- Allow for serialization of ``netcdftime.datetime`` objects (:issue:`789`,
+  :issue:`1084`, :issue:`1252`). By `Spencer Clark
+         <https://github.com/spencerkclark>`_.
  - Some speed improvement to construct :py:class:`~xarray.DataArrayRolling`
  object (:issue:`1993`)
  By `Keisuke Fujii <https://github.com/fujiisoup>`_.

xarray/__init__.py

@@ -19,7 +19,6 @@
                            save_mfdataset)
 from .backends.rasterio_ import open_rasterio
 from .backends.zarr import open_zarr
-
 from .conventions import decode_cf, SerializationWarning
 
 try:

xarray/backends/zarr.py

@@ -8,6 +8,8 @@
 
 from .. import Variable, coding, conventions
 from ..core import indexing
+from ..core.common import (contains_netcdftime_datetimes,
+                           contains_datetime_datetimes)
 from ..core.pycompat import OrderedDict, integer_types, iteritems
 from ..core.utils import FrozenOrderedDict, HiddenKeyDict
 from .common import AbstractWritableDataStore, ArrayWriter, BackendArray
@@ -221,7 +223,8 @@ def encode_zarr_variable(var, needs_copy=True, name=None):
         A variable which has been encoded as described above.
     """
 
-    if var.dtype.kind == 'O':
+    if var.dtype.kind == 'O' and not (contains_netcdftime_datetimes(var) or
+                                      contains_datetime_datetimes(var)):
         raise NotImplementedError("Variable `%s` is an object. Zarr "
                                   "store can't yet encode objects." % name)