Skip to content

Latest commit

 

History

History
139 lines (126 loc) · 6.1 KB

README.NetCDF4.md

File metadata and controls

139 lines (126 loc) · 6.1 KB
Raw

Support NetCDF-4 files

Starting from version 1.11.0, PnetCDF supports data access to HDF5-based NetCDF-4 files. This format is also referred as the enhanced data model, versus the classic model (in NetCDF language). The new I/O driver, called the NetCDF-4 driver, is a wrapper of the NetCDF Library that translates PnetCDF APIs to their corresponding NetCDF APIs, for example, ncmpi_create calls nc_create. Through calling PnetCDF APIs, this feature allows applications to read/write files in NetCDF-4 format.

Enable NetCDF-4 support

PnetCDF requires a NetCDF library that is built with parallel HDF5 capability. Example build instructions for HDF5 and NetCDF-4 are given below.

  • To build parallel HDF5 (version 1.10.4 or later is recommended)
    • Source tar ball of HDF5 can be downloaded from URL: https://www.hdfgroup.org/downloads/hdf5/source-code/
    • Build commands: 
      gzip -dc hdf5-1.10.4.tar.gz | tar -xf -
cd hdf5-1.10.4
./configure --prefix=/HDF5/install/path \
            --enable-parallel=yes \
            CC=mpicc FC=mpifort CXX=mpicxx
make install
  • To build NetCDF-4 (version 4.6.2 or later is recommended)
    • Source tar ball of NetCDF-4 can be downloaded from URL: https://github.com/Unidata/netcdf-c/releases
    • Build commands: 
      gzip -dc v4.6.2.tar.gz | tar -xf -
cd netcdf-c-4.6.2
./configure --prefix=/NetCDF4/install/path \
            --enable-netcdf-4 \
            CC=mpicc \
            CPPFLAGS="-I/HDF5/install/path/include" \
            LDFLAGS="-L/HDF5/install/path/lib"
make install
    • Note NetCDF-4 library built with PnetCDF feature enabled, i.e. option --enable-pnetcf, is not supported.
  • To build PnetCDF with NetCDF-4 support
    • Add --with-netcdf4 option at the configure command line. Option --with-netcdf can also be used to specify the installation path of NetCDF-4 if it is not in the standard localtions. For example, 
      gzip -dc pnetcdf-1.11.0.tar.gz | tar -xf -
cd pnetcdf-1.11.0
./configure --prefix=/PnetCDF/install/path \
            --with-netcdf4=/NetCDF4/install/path

Accessing NetCDF-4 file

To create a NetCDF-4 file, add the flag NC_NETCDF4 into argument cmode when calling ncmpi_create(). For example,

int cmode;
cmode = NC_CLOBBER | NC_NETCDF4;
ncmpi_create(MPI_COMM_WORLD, "testfile.nc", cmode, MPI_INFO_NULL, &ncid);

or

cmode = NC_CLOBBER | NC_NETCDF4 | NC_CLASSIC_MODEL;
ncmpi_create(MPI_COMM_WORLD, "testfile.nc", cmode, MPI_INFO_NULL, &ncid);

NC_NETCDF4 flag is not required when opening an existing file in NetCDF-4 format, as PnetCDF checks the file format and selects the proper I/O driver.

Users can also set the default file format to NetCDF-4 by calling API ncmpi_set_default_format() with argument NC_FORMAT_NETCDF4C or NC_FORMAT_NETCDF4_CLASSIC. For example,

ncmpi_set_default_format(NC_FORMAT_NETCDF4, &old_formatp);

or

ncmpi_set_default_format(NC_FORMAT_NETCDF4_CLASSIC, &old_formatp);

When no file format specific flag is set in argument cmode, PnetCDF will use the default setting.

Known Problems

Some features are not supported due to the availability of APIs different between PnetCDF and NetCDF libraries. I/O semantics are also slightly different.

  • API families of vard, varn, and nonblocking I/O are not supported. This is because NetCDF does not have corresponding APIs. Error code NC_ENOTSUPPORT will be returned. For vard APIs, NetCDF does not have APIs that allow accessing the file directly by an MPI derived file type. For varn APIs, a potential solution is to split the request into into multiple vara calls. However, such solution must deal with the situation when the numbers of requests are different among processes in the collective data mode. Supporting varn is thus a future work.
  • Although a new file of format NC_FORMAT_NETCDF4 or NC_FORMAT_NETCDF4_CLASSIC can be created, the I/O operations are limited to the NetCDF classic model I/O, This is because PnetCDF APIs do not include those for operating enhanced data objects, such as groups, compound data types, compression etc. As for reading a NetCDF-4 file created by NetCDF-4, PnetCDF supports the classic way of reading variables, attributes, and dimensions. Inquiry APIs for enhanced metadata is currently not supported.
  • Due to a bug in HDF5 version 1.10.2 and prior, collective I/O on record variables with a subset of participating processes making zero-length requests may cause an HDF5 error and the program to hang. Readers refer to the HDF bug issue HDFFV-10501. The bug fix will appear in HDF5 1.10.4 release.
  • PnetCDF allows different kinds of APIs called by different processes in a collective I/O. For example, ncmpi_put_vara_int_all() is called at process 0 and ncmpi_put_vars_float_int() is called at process 1. However, this is not allowed in NetCDF-4. Thus, the same kind of API must be used in a collective call when accessing a NetCDF-4 file.
  • When creating a new NetCDF-4 file, PnetCDF only allows creating 1 unlimited dimension. When reading an existing NetCDF-4 file, the file can have more than 1 unlimited dimension. However, in this case, API ncmpi_inq_unlimdim will only return the dimension ID of first unlimited dimension. This behavior conforms with NetCDF-4 library.
  • The following APIs are not supported yet. An error code NC_ENOTSUPPORT will be returned, if called.
    • ncmpi_inq_header_size, ncmpi_inq_header_extent
    • ncmpi_inq_striping
    • ncmpi_inq_varoffset
    • ncmpi_fill_var_rec
    • ncmpi_sync_numrecs
    • ncmpi_flush
    • all nonblocking APIs
    • vard and varn APIs
    • flexible APIs (i.e. argument buftype is a constructed MPI derived data type)
  • APIs ncmpi_inq_get_size and ncmpi_inq_put_size report the amount of data that has been read or written, but excluding the I/O to the file header. They are simply the size of data passed between PnetCDF and NetCDF, not the actual size read from or written to the file system.

Copyright (C) 2018, Northwestern University and Argonne National Laboratory. See COPYRIGHT notice in top-level directory.