NetCDF library configuration

[gsjaardema]

The configuration of the NetCDF library, specifically for parallel builds, is becoming more complex and can cause build issues or reduced functionality if configured incorrectly. For example:

• NetCDF can be built with or without HDF5 support. If built without HDF5, then the compression and netcdf-4 parallel support isn't available
  • NetCDF can be built with or without parallel-netcdf (pnetcdf) support. If built without pnetcdf, then parallel capabilities for netcdf-3 files is not enabled.
  • Exodus requires that NC_MAX_DIMS and NC_MAX_VARS be increased from their default values. If not changed, then some medium complex models cannot be created or read (See https://github.com/gsjaardema/seacas/blob/master/NetCDF-Mapping.md)
  • NetCDF can be built with or without parallel support.

Currently, a developer will build HDF5 with its options; build parallel-netcdf if needed; and then build netcdf -- hopefully pointing it to the correct hdf5 and parallel-netcdf and setting the correct configure options. Then, the developer will configure Trilinos and has to also specify some of these options correctly so that the Exodus library in the SEACAS package will build with the correct capabilities.

This is error-prone and can be difficult to determine what part of the build/configuration was done incorrectly. The stand-alone seacas project (https://github.com/gsjaardema/seacas.git) has some cmake modules that will query the installed NetCDF library and determine whether it was built with no-hdf5, serial hdf5, or parallel hdf5; whether it was built with or without parallel-netcdf; and whether it has the correct values for NC_MAX_DIMS and NC_MAX_VARS. The module then sets the correct cmake variables.

These cmake modules are in the cmake/modules in the seacas github repository referenced above.

I would like to propose using these modules, or a variant of them in Trilinos to reduce the complexity of configuring the NetCDF and Exodus libraries.
@trilinos/seacas

[jwillenbring]

@bmpersc @bartlettroscoe

Adding Brent and Ross to the discussion.

[bartlettroscoe]

Someone should really start putting together build scripts for Trilinos TPLs. I would say that Spack would be the best option to start looking into.

[jgfouca]

@bartlettroscoe , we're already doing that on the SEMS side.

[bartlettroscoe]

@bartlettroscoe , we're already doing that on the SEMS side.

Great! Can you help @gsjaardema out?

[jgfouca]

We can offer pre-built hdf5/netcdf for a set of compilers and MPIs

[gdsjaar]

I'm not sure I (@gsjaardema) need help. I was hoping to add some code to Tribits that would help others who are configuring netcdf and other libraries required/optional for the seacas product. The scripts that I use in the standalone github seacas build take care of setting some options that must be done manually in the Trilinos configuration.

For example, in seacas it is automatically deteremined whether netcdf is built with hdf5 (netcdf-4) and pnetcdf libraries as dependencies; similarly for matio and cgns. This must be manually specified in Trilinos configure.

The scripts also check for the netcdf modifications highly-recommended for use with exodus.
..Greg

"A supercomputer is a device for turning compute-bound problems into I/O-bound problems"

From: "Roscoe A. Bartlett" <[email protected]mailto:[email protected]>
Reply-To: trilinos/Trilinos <[email protected]mailto:[email protected]>
Date: Tuesday, February 2, 2016 at 2:24 PM
To: trilinos/Trilinos <[email protected]mailto:[email protected]>
Subject: [EXTERNAL] Re: [Trilinos] NetCDF library configuration (#99)

@bartlettroscoehttps://github.com/bartlettroscoe , we're already doing that on the SEMS side.

Great! Can you help @gsjaardemahttps://github.com/gsjaardema out?

Reply to this email directly or view it on GitHubhttps://github.com//issues/99#issuecomment-178829025.

[jgfouca]

Ah, sorry I misunderstood what you were asking for.

[bartlettroscoe]

We can offer pre-built hdf5/netcdf for a set of compilers and MPIs

That is fine for SNL systems but that does nothing to help others not on an SNL system.

Is anyone working on a set of portable build wrappers for these TPLs like exists for Spack:

https://github.com/LLNL/spack

or like we and other have done for CASL VERA with CMake ExternalProject? I know this is a lot of work, especially to make these portable but that is what Trilinos (and downstream APPs) needs. That is what PETSc has been doing for a long time.

[bartlettroscoe]

I was hoping to add some code to Tribits that would help others who are configuring netcdf and other libraries required/optional for the seacas product

@gsjaardema, once I get fully implement TriBITS #63, then it will be possible to allow Trilinos (or SEACAS or any other TriBITS project) users to build upstream TPLs like BLAS, LAPACK , SuperLU, HDF5, NetCDF just like existing TriBITS packages. But someone will still need to figure out a portable way to build these TPLs from source on various systems. TriBITS can't magically do that. That is very hard to do in my experience.

[gsjaardema]

I worded that incorrectly. What I meant was to help others who are
configuring Trilinos based on the external netcdf, hdf5, matio, pnetcdf,
and cgns libraries. The code in seacas will query those external libraries
to determine, for example, whether the netcdf library was built with hdf5
and pnetcdf support and set the appropriate seacas or trilinos defines
appropriately. The user still needs to build the external libraries
outside of seacas or Trilinos.

On Tue, Feb 2, 2016 at 2:39 PM, Roscoe A. Bartlett <[email protected]
javascript:_e(%7B%7D,'cvml','[email protected]');> wrote:

I was hoping to add some code to Tribits that would help others who are
configuring netcdf and other libraries required/optional for the seacas
product

@gsjaardema https://github.com/gsjaardema, once I get fully implement TriBITS
#63 TriBITSPub/TriBITS#63, then it will be
possible to allow Trilinos (or SEACAS or any other TriBITS project) users
to build upstream TPLs like BLAS, LAPACK , SuperLU, HDF5, NetCDF just like
existing TriBITS packages. But someone will still need to figure out a
portable way to build these TPLs from source on various systems. TriBITS
can't magically do that. That is very hard to do in my experience.

—
Reply to this email directly or view it on GitHub
#99 (comment).

[bartlettroscoe]

The code in seacas will query those external libraries to determine, for example, whether the netcdf library was built with hdf5 and pnetcdf support and set the appropriate seacas or trilinos defines appropriately.

@gsjaardema, you should be able to do that with configure-time checks in the SEACAS package (or the appropriate SEACAS subpackage). You can see examples of configure-time tests like this in Teuchos.

[gsjaardema]

This should be fixed with the adding of the improved NetCDF package find script in TriBITs