Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

broken output of find_root_and_group for h5netcdf #3680

Closed
kmuehlbauer opened this issue Jan 10, 2020 · 4 comments
Closed

broken output of find_root_and_group for h5netcdf #3680

kmuehlbauer opened this issue Jan 10, 2020 · 4 comments

Comments

@kmuehlbauer
Copy link
Contributor

kmuehlbauer commented Jan 10, 2020

MCVE Code Sample

# Your code here
import netCDF4
import h5netcdf
import xarray
from xarray.backends.common import find_root_and_group
root = netCDF4.Dataset("test.nc", "w", format="NETCDF4")
test1 = root.createGroup("test1")
test1.createGroup("test2")
root.close()
h5n = h5netcdf.File('test.nc')
print(find_root_and_group(h5n['test1'])[1])
h5n.close()

This will output:
//test1//test1/test2

Expected Output

/test1/test2

Problem Description

I stumbled over this while working on #3618. Although the function claims to retrieve root and group name of netCDF4/h5netcdf datasets the output for h5netcdf is broken.

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.8.0 | packaged by conda-forge | (default, Nov 22 2019, 05:03:59)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 4.12.14-lp151.28.36-default
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: de_DE.UTF-8
LOCALE: de_DE.UTF-8
libhdf5: 1.10.5
libnetcdf: 4.7.3

xarray: 0.14.1+43.g3e9d9046.dirty
pandas: 0.25.3
numpy: 1.17.3
scipy: 1.3.3
netCDF4: 1.5.3
pydap: None
h5netcdf: 0.7.4
h5py: 2.10.0
Nio: None
zarr: None
cftime: 1.0.4.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2.8.0
distributed: 2.9.0
matplotlib: 3.1.2
cartopy: 0.17.0
seaborn: None
numbagg: None
setuptools: 42.0.2.post20191201
pip: 19.3.1
conda: None
pytest: 5.3.1
IPython: 7.10.2
sphinx: 2.3.1

@kmuehlbauer
Copy link
Contributor Author

Good chance that this might be an h5netcdf issue. Can someone double check?

@kmuehlbauer
Copy link
Contributor Author

After a bit of further investigation, looks like this is indeed a problem with how h5netcdf retrieves the .name. While netCDF4 gives back just the name of a group h5netcdf gives back the hdf5 name which is the full path to the group.

# Your code here
import netCDF4
import h5netcdf
import xarray
from xarray.backends.common import find_root_and_group
root = netCDF4.Dataset("testx.nc", "w", format="NETCDF4")
test1 = root.createGroup("test1")
test1.createGroup("test2")
print(root['test1/test2'].name)
root.close()
h5n = h5netcdf.File('testx.nc')
print(h5n['test1/test2'].name)
h5n.close()

Output:

test2
/test1/test2

kmuehlbauer added a commit to kmuehlbauer/xarray that referenced this issue Jan 13, 2020
kmuehlbauer added a commit to kmuehlbauer/xarray that referenced this issue Jan 13, 2020
shoyer pushed a commit that referenced this issue Jan 20, 2020
#3618)

* ENH: enable `H5NetCDFStore` to work with already open h5netcdf.File and h5netcdf.Group objects, add test

* FIX: add `.open` method for file-like objects

* FIX: reformat using black

* WIP: add item to whats-new.rst

* FIX: temporary fix to tackle issue #3680

* FIX: do not use private API, use find_root_and_group instead

* FIX: reformat using black
@kmuehlbauer
Copy link
Contributor Author

kmuehlbauer commented Jan 23, 2020

temporarily fixed by #3618 . This is now tracked in h5netcdf/h5netcdf#68.

@kmuehlbauer
Copy link
Contributor Author

A bit more of context:

As it turns out, the temporary fix will be permanent for the time being. This is because h5netcdf aligns with h5py(returning full path) for it's new API. The legacyapi (netCDF4 drop-in) will return single group name (not full path).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant