How to convert cftime.Datetime360Day() object to python datetime?

[ru111]

I've loaded data in xarray format which has a coordinate 'time'. The first 4 values look like:

array([cftime.Datetime360Day(1970, 1, 1, 12, 0, 0, 0, 2, 1),
       cftime.Datetime360Day(1970, 1, 2, 12, 0, 0, 0, 3, 2),
       cftime.Datetime360Day(1970, 1, 3, 12, 0, 0, 0, 4, 3),
       cftime.Datetime360Day(1970, 1, 4, 12, 0, 0, 0, 5, 4)], dtype=object)

I'm not quite sure how to interpret all of these numbers (1970, 1, 1, 12, 0, 0, 0, 2, 1) (I couldn't find it in the documentation) but assuming the first 3 digits is year-month-day, I would like to convert this array into a numpy datetime64[nc] format, or some other general datetime format (like pandas datetime), something like:

DatetimeIndex(['1979-01-01', '1979-01-02', '1979-01-03', '1979-01-04',
               '1979-01-05'],
              dtype='datetime64[ns]', freq='D')

I want to do this because I can't use the xarray .plot() function to view the timeseries since it doesn't seem to understand the cftime format. I would like to get all the datetimes of all of my datasets in the same format because others have datetime64[nc], cftime.DatetimeNoLeap() etc. and I would like to standardise my datetime format to make my analysis easier...

[jswhit]

You can't convert it into a python datetime object, since python datetime does not support that calendar. The Datetime360day object has many of the same attributes and methods as the python datetime object (including year,month,day,hour,minute,second,microsecond) and supports timedelta operations.

It would be nice to know what xarray.plot is expecting that Datetime360day doesn't have - perhaps this could be fixed.

[ru111]

Many thanks for your quick reply. There may not be a function to convert it, but could I write something that can extract the year, month and day from the cftime calendar? Then I can manually concatenate it to create a list ['1979-01-01', '1979-01-02', '1979-01-03' ...] then convert it to a python datatime and perhaps approximate or eliminate the dates that are not possible python datatime (if by "python datatime does not support that calendar" you mean that cftime will sometimes contain dates that do not exist/are not possible in python datatime?). I have other xarray datasets that have the time coordinate in python datetime format so being able to convert from cftime would be extremely handy. Or perhaps there is a way to convert python datatime to a cftime format? I would like to know if there is.

xarray plotting error gives the following: TypeError: Plotting requires coordinates to be numeric or dates of type np.datetime64 or datetime.datetime or pd.Interval.

As a workaround it would be nice to know if there was a method to manipulate these calendars to force the time to 0:0:0:0 where just the date is important (so that I can concatenate two data arrays to a dataset with one data array having a different time stamp to the other for no reason), so that:

array([cftime.Datetime360Day(1970, 1, 1, 12, 0, 0, 0, 2, 1),
       cftime.Datetime360Day(1970, 1, 2, 12, 0, 0, 0, 3, 2),
       cftime.Datetime360Day(1970, 1, 3, 12, 0, 0, 0, 4, 3),
       cftime.Datetime360Day(1970, 1, 4, 12, 0, 0, 0, 5, 4)], dtype=object)

becomes:

array([cftime.Datetime360Day(1970, 1, 1, 0, 0, 0, 0, 0, 0),
       cftime.Datetime360Day(1970, 1, 2, 0, 0, 0, 0, 0, 0),
       cftime.Datetime360Day(1970, 1, 3, 0, 0, 0, 0, 0, 0),
       cftime.Datetime360Day(1970, 1, 4, 0, 0, 0, 0, 0, 0)], dtype=object)

So that at least I can combine arrays with the same cftime.x formats.

[spencerkclark]

It would be nice to know what xarray.plot is expecting that Datetime360day doesn't have - perhaps this could be fixed.

We fixed this very recently in xarray by adding an optional dependency on nc-time-axis, a package that enables plotting cftime dates in matplotlib. The changes will take effect in the next version, which has yet to be released (version 0.12.0).

Until then, @ru111, for plotting you have a couple options:

• My recommended solution (if you are only concerned with plotting) would be to install nc-time-axis immediately and drop your data into matplotlib directly, rather than use xarray's built-in plotting for the time being.
• Or you can have xarray try to convert your dates to np.datetime64[ns] using to_datetimeindex (see the example at the bottom of this page). However, as @jswhit notes the calendar of the cftime.Datetime360Day date type is not strictly compatible with np.datetime64 (e.g. timedelta operations may no longer be accurate, and certain dates in a 360-day calendar are not possible to represent in a proleptic Gregorian calendar, the calendar type for np.datetime64, like February 30th).

[spencerkclark]

Note with the latest version of nc-time-axis, you can plot with cftime dates directly (rather than converting to CalendarDateTime objects, as suggested in the README). See an example here.

[spencerkclark]

As a workaround it would be nice to know if there was a method to manipulate these calendars to force the time to 0:0:0:0 where just the date is important (so that I can concatenate two data arrays to a dataset with one data array having a different time stamp to the other for no reason), so that:

This seems somewhat orthogonal to plotting and could be worth bringing up as an issue in the xarray repo. In pandas speak this is called "normalization" -- see pandas.DatetimeIndex.normalize. Internally we have a method that normalizes cftime dates, but it is not currently available publicly. It would be fairly straightforward to write a xarray.CFTimeIndex.normalize method which would behave just as the pandas version does, which would allow you to normalize dates as a group.

[jswhit]

Does the nc-time-axis feature your problem @ru111 ? If so, I will close this issue.

[ru111]

It would be nice to know what xarray.plot is expecting that Datetime360day doesn't have - perhaps this could be fixed.

We fixed this very recently in xarray by adding an optional dependency on nc-time-axis, a package that enables plotting cftime dates in matplotlib. The changes will take effect in the next version, which has yet to be released (version 0.12.0).

Until then, @ru111, for plotting you have a couple options:

• My recommended solution (if you are only concerned with plotting) would be to install nc-time-axis immediately and drop your data into matplotlib directly, rather than use xarray's built-in plotting for the time being.
• Or you can have xarray try to convert your dates to np.datetime64[ns] using to_datetimeindex (see the example at the bottom of this page). However, as @jswhit notes the calendar of the cftime.Datetime360Day date type is not strictly compatible with np.datetime64 (e.g. timedelta operations may no longer be accurate, and certain dates in a 360-day calendar are not possible to represent in a proleptic Gregorian calendar, the calendar type for np.datetime64, like February 30th).

Exactly the solution I was looking for - thanks @spencerkclark !