Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calendar utilities #5233

Merged
merged 45 commits into from
Dec 30, 2021
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
3e72df9
dt.calendar and date_range
aulemahal Apr 21, 2021
1c37bbd
Migrate calendar utils from xclim | add dt.calendar
aulemahal Apr 28, 2021
1ffd74c
Merge remote-tracking branch 'upstream/master' into calendar-utils
aulemahal Apr 28, 2021
39079e3
upd whats new
aulemahal Apr 28, 2021
11d15ee
skip calendar tests with no cftime
aulemahal Apr 28, 2021
d8ec022
add requires cftime 1.1.0
aulemahal Apr 28, 2021
8fe0a94
import date_ranges in main
aulemahal Apr 28, 2021
f47f823
Apply suggestions from code review
aulemahal Apr 30, 2021
c58e2ae
Merge remote-tracking branch 'upstream/master' into calendar-utils
aulemahal Apr 30, 2021
c311002
Add docs - use already existing is np datetime func
aulemahal Apr 30, 2021
10cf483
Merge remote-tracking branch 'upstream/master' into calendar-utils
aulemahal May 7, 2021
d9e174a
update from suggestions
aulemahal May 7, 2021
84ebc89
Merge remote-tracking branch 'upstream/master' into calendar-utils
aulemahal May 17, 2021
9d6254b
Apply suggestions from code review
aulemahal May 17, 2021
976b3cf
Merge branch 'calendar-utils' of https://github.com/aulemahal/xarray …
aulemahal May 17, 2021
0fce9cb
Modifications following review
aulemahal May 17, 2021
aa74140
Add DataArray and Dataset methods
aulemahal May 17, 2021
bc7a912
use proper type annotation
aulemahal May 17, 2021
5aa9732
Apply suggestions from code review
aulemahal Aug 13, 2021
ca566bd
some more modifications after review
aulemahal Aug 13, 2021
2d7201f
merge main
aulemahal Aug 13, 2021
f307834
merge main
aulemahal Aug 31, 2021
a3e9fb2
Apply suggestions from code review
aulemahal Aug 31, 2021
97909f7
Finish applying suggestions from review
aulemahal Aug 31, 2021
507c501
Put back missing @require_cftime
aulemahal Aug 31, 2021
599882f
Merge branch 'main' into calendar-utils
aulemahal Sep 23, 2021
44be4e5
Apply suggestions from code review
aulemahal Sep 23, 2021
aa03268
Merge branch 'calendar-utils' of github.com:aulemahal/xarray into cal…
aulemahal Sep 23, 2021
c4570d8
Add tests - few fixes
aulemahal Sep 23, 2021
92ab8ba
Merge branch 'main' into calendar-utils
aulemahal Sep 29, 2021
2088230
wrap docstrings
aulemahal Sep 29, 2021
d5b50dc
Change way of importing/testing for cftime
aulemahal Sep 29, 2021
dc9338e
Upd the weather-climate doc page
aulemahal Sep 29, 2021
822529c
fix doc examples
aulemahal Sep 29, 2021
b86de04
Neat docs
aulemahal Sep 30, 2021
d7efe8e
fix in tests after review
aulemahal Oct 12, 2021
4430350
Apply suggestions from code review
aulemahal Oct 18, 2021
790be22
Better explain missing in notes - copy changes to obj methods
aulemahal Oct 18, 2021
eb96222
Merge branch 'main' into calendar-utils
aulemahal Oct 18, 2021
0726082
Merge branch 'main' into calendar-utils
aulemahal Oct 26, 2021
b6f53a8
Apply suggestions from code review
aulemahal Oct 26, 2021
4f02ca7
Merge branch 'calendar-utils' of github.com:aulemahal/xarray into cal…
aulemahal Oct 26, 2021
0f8d5d5
Merge branch 'main' into calendar-utils
aulemahal Dec 13, 2021
2c023a4
Remove unused import
aulemahal Dec 13, 2021
5aa7470
Merge branch 'main' into pr/5233
Illviljan Dec 29, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/api-hidden.rst
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,7 @@
core.accessor_dt.DatetimeAccessor.floor
core.accessor_dt.DatetimeAccessor.round
core.accessor_dt.DatetimeAccessor.strftime
core.accessor_dt.DatetimeAccessor.calendar
core.accessor_dt.DatetimeAccessor.date
core.accessor_dt.DatetimeAccessor.day
core.accessor_dt.DatetimeAccessor.dayofweek
Expand Down
7 changes: 7 additions & 0 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,8 @@ Dataset contents
Dataset.drop_dims
Dataset.set_coords
Dataset.reset_coords
Dataset.convert_calendar
Dataset.interp_calendar

Comparisons
-----------
Expand Down Expand Up @@ -296,6 +298,8 @@ DataArray contents
DataArray.drop_duplicates
DataArray.reset_coords
DataArray.copy
DataArray.convert_calendar
DataArray.interp_calendar

**ndarray methods**:
:py:attr:`~DataArray.astype`
Expand Down Expand Up @@ -514,6 +518,7 @@ Datetimelike properties
DataArray.dt.season
DataArray.dt.time
DataArray.dt.date
DataArray.dt.calendar
aulemahal marked this conversation as resolved.
Show resolved Hide resolved
DataArray.dt.is_month_start
DataArray.dt.is_month_end
DataArray.dt.is_quarter_end
Expand Down Expand Up @@ -842,6 +847,8 @@ Creating custom indexes
:toctree: generated/

cftime_range
date_range
date_range_like

Faceting
--------
Expand Down
45 changes: 20 additions & 25 deletions doc/user-guide/weather-climate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,23 @@ using the same formatting as the standard `datetime.strftime`_ convention .
dates.strftime("%c")
da["time"].dt.strftime("%Y%m%d")

Conversion between non-standard calendar and to/from pandas DatetimeIndexes is
facilitated with the :py:meth:`xarray.Dataset.convert_calendar` method (also available as
:py:meth:`xarray.DataArray.convert_calendar`). Here, like elsewhere in xarray, the ``use_cftime``
argument controls which datetime backend is used in the output. The default (``None``) is to
use `pandas` when possible, i.e. when the calendar is standard and dates are within 1678 and 2262.

.. ipython:: python

dates = xr.cftime_range(start="2001", periods=24, freq="MS", calendar="noleap")
da_nl = xr.DataArray(np.arange(24), coords=[dates], dims=["time"], name="foo")
da_std = da.convert_calendar("standard", use_cftime=True)

The data is unchanged, only the timestamps are modified. Further options are implemented
for the special ``"360_day"`` calendar and for handling missing dates. There is also
:py:meth:`xarray.Dataset.interp_calendar` (and :py:meth:`xarray.DataArray.interp_calendar`)
for `interpolating` data between calendars.

For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:

- `Partial datetime string indexing`_:
Expand All @@ -150,7 +167,8 @@ For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:

- Access of basic datetime components via the ``dt`` accessor (in this case
just "year", "month", "day", "hour", "minute", "second", "microsecond",
"season", "dayofyear", "dayofweek", and "days_in_month"):
"season", "dayofyear", "dayofweek", and "days_in_month") with the addition
of "calendar", absent from pandas:

.. ipython:: python

Expand All @@ -160,6 +178,7 @@ For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:
da.time.dt.dayofyear
da.time.dt.dayofweek
da.time.dt.days_in_month
da.time.dt.calendar

- Rounding of datetimes to fixed frequencies via the ``dt`` accessor:

Expand Down Expand Up @@ -214,30 +233,6 @@ For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:

da.resample(time="81T", closed="right", label="right", base=3).mean()

.. note::


For some use-cases it may still be useful to convert from
a :py:class:`~xarray.CFTimeIndex` to a :py:class:`pandas.DatetimeIndex`,
despite the difference in calendar types. The recommended way of doing this
is to use the built-in :py:meth:`~xarray.CFTimeIndex.to_datetimeindex`
method:

.. ipython:: python
:okwarning:

modern_times = xr.cftime_range("2000", periods=24, freq="MS", calendar="noleap")
da = xr.DataArray(range(24), [("time", modern_times)])
da
datetimeindex = da.indexes["time"].to_datetimeindex()
da["time"] = datetimeindex

However in this case one should use caution to only perform operations which
do not depend on differences between dates (e.g. differentiation,
interpolation, or upsampling with resample), as these could introduce subtle
and silent errors due to the difference in calendar types between the dates
encoded in your data and the dates stored in memory.

.. _Timestamp-valid range: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#timestamp-limitations
.. _ISO 8601 standard: https://en.wikipedia.org/wiki/ISO_8601
.. _partial datetime string indexing: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#partial-string-indexing
2 changes: 2 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ New Features
- Added ``storage_options`` argument to :py:meth:`to_zarr` (:issue:`5601`).
By `Ray Bell <https://github.com/raybellwaves>`_, `Zachary Blackwood <https://github.com/blackary>`_ and
`Nathan Lis <https://github.com/wxman22>`_.
- Added calendar utilities :py:func:`DataArray.convert_calendar`, :py:func:`DataArray.interp_calendar`, :py:func:`date_range`, :py:func:`date_range_like` and :py:attr:`DataArray.dt.calendar`. (:pull:`5233`).
aulemahal marked this conversation as resolved.
Show resolved Hide resolved
By `Pascal Bourgault <https://github.com/aulemahal>`_.
- Histogram plots are set with a title displaying the scalar coords if any, similarly to the other plots (:issue:`5791`, :pull:`5792`).
By `Maxime Liquet <https://github.com/maximlt>`_.

Expand Down
4 changes: 3 additions & 1 deletion xarray/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
)
from .backends.rasterio_ import open_rasterio
from .backends.zarr import open_zarr
from .coding.cftime_offsets import cftime_range
from .coding.cftime_offsets import cftime_range, date_range, date_range_like
from .coding.cftimeindex import CFTimeIndex
from .coding.frequencies import infer_freq
from .conventions import SerializationWarning, decode_cf
Expand Down Expand Up @@ -52,6 +52,8 @@
"combine_by_coords",
"combine_nested",
"concat",
"date_range",
"date_range_like",
"decode_cf",
"dot",
"cov",
Expand Down
Loading