Decode time error in CESM POP output

[Yefee]

Hi, all

Recently, I found a error about time decoding. The .nc file is POP output.

In [29]: ds = xr.open_mfdataset('EXAMPLE_CASE.pop.h.0001-01.nc')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-29-576c2580c841> in <module>()
----> 1 ds = xr.open_mfdataset('EXAMPLE_CASE.pop.h.0001-01.nc')

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/backends/api.pyc in open_mfdataset(paths, chunks, concat_dim, preprocess, engine, lock, **kwargs)
    300         lock = _default_lock(paths[0], engine)
    301     datasets = [open_dataset(p, engine=engine, chunks=chunks or {}, lock=lock,
--> 302                              **kwargs) for p in paths]
    303     file_objs = [ds._file_obj for ds in datasets]
    304

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/backends/api.pyc in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, concat_characters, decode_coords, engine, chunks, lock, drop_variables)
    225             lock = _default_lock(filename_or_obj, engine)
    226         with close_on_error(store):
--> 227             return maybe_decode_store(store, lock)
    228     else:
    229         if engine is not None and engine != 'scipy':

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/backends/api.pyc in maybe_decode_store(store, lock)
    156             store, mask_and_scale=mask_and_scale, decode_times=decode_times,
    157             concat_characters=concat_characters, decode_coords=decode_coords,
--> 158             drop_variables=drop_variables)
    159
    160         if chunks is not None:

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in decode_cf(obj, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables)
    888     vars, attrs, coord_names = decode_cf_variables(
    889         vars, attrs, concat_characters, mask_and_scale, decode_times,
--> 890         decode_coords, drop_variables=drop_variables)
    891     ds = Dataset(vars, attrs=attrs)
    892     ds = ds.set_coords(coord_names.union(extra_coords))

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in decode_cf_variables(variables, attributes, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables)
    823         new_vars[k] = decode_cf_variable(
    824             v, concat_characters=concat, mask_and_scale=mask_and_scale,
--> 825             decode_times=decode_times)
    826         if decode_coords:
    827             var_attrs = new_vars[k].attrs

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in decode_cf_variable(var, concat_characters, mask_and_scale, decode_times, decode_endianness)
    764             units = pop_to(attributes, encoding, 'units')
    765             calendar = pop_to(attributes, encoding, 'calendar')
--> 766             data = DecodedCFDatetimeArray(data, units, calendar)
    767         elif attributes['units'] in TIME_UNITS:
    768             # timedelta

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in __init__(self, array, units, calendar)
    389             if not PY3:
    390                 msg += ' Full traceback:\n' + traceback.format_exc()
--> 391             raise ValueError(msg)
    392         else:
    393             self._dtype = getattr(result, 'dtype', np.dtype('object'))

ValueError: unable to decode time units u'days since 0000-01-01 00:00:00' with the default calendar. Try opening your dataset with decode_times=False.`

The actual time is:

  31     double time(time) ;

  32         time:long_name = "time" ;

  33         time:units = "days since 0000-01-01 00:00:00" ;

  34         time:bounds = "time_bound" ;

  35         time:calendar = "noleap" ;

I can set 'decode_times=False' to open the file but the time is not right.

Coordinates:
    transport_components  (transport_comp) |S256 'Total' ...
    transport_regions     (transport_reg) |S256 'Global Ocean - Marginal Seas' ...
  * time                  (time) float64 396.0```

Is there any suggestions?

[Yefee]

A similar issue is found here: #521
sloutions: #521

[rabernat]

@Yefee I wouldn't really say this is "solved". You can bypass the date decoding using decode_times=False, but then you lose lots of useful functionality (e.g. resampling). As described in #521, truly solving the problem will require upstream fixes in numpy and pandas. This is a persistent and highly frustrating problem for the xarray community.

[Yefee]

Thanks for your comment. @rabernat
The time decoding problem is solved to some extent for me. However,I have another question now. That is how to deal with the coordinates in pop output. Most of the variables have coordinates like (time, nlat, nlon), but they are not the real coordinates in space. If I combine xarray and cartopy to plot something, (i.e. temperature) the nlat and nlon coordinates is annoying. Do you have any good ideas?

[jhamman]

@Yefee - we try to keep discussion topics on github to a single issue. This issue is about decoding the time coordinate in your dataset. If you have other questions (e.g. plotting), it would help us if you used one of the following forums to ask it:

• our user mailing list: https://groups.google.com/forum/#!forum/xarray
• the xarray stack overflow tag: http://stackoverflow.com/questions/tagged/python-xarray
• a new github issue: https://github.com/pydata/xarray/issues/new

[jhamman]

I don't think we need another calendar/datetime issue so I'm going to close this. @Yefee, if you're interested in following along with the progress in this area, see #789.