Implement PIOc_open and PIOc_create functions

[edhartnett]

As discussed with Jim and Kate, we should add PIO wrappers to nc_open/nc_create.

Not only would this be more obvious for the netCDF user, it would address certain problems with the current PIO methods of creating and opening files:

• Create mode can be used to create all netCDF binary formats without changing the PIO API. That is, when the netCDF team adds a new output format (like CDF5), the PIO library will be able to transparently handle the new format.
• Very naughty practice of re-opening files in netCDF serial if parallel fails can be supported in existing functions, but not carried forward. This practice, while convenient to some important and valued users, will lead to many problem. Failure of parallel IO on expensive new hardware will not be noticed. Performance may suffer and it will be very confusing.
• Other settings passed in the netCDF modes will be transparently supported as well, for example read-only.

[jedwards4b]

I'm not sure that I understand your second comment. I believe that we log a message to standard out if a failover to serial happens and failure of parallel IO should be pretty noticeable especially on platforms where parallel IO performs well.

[edhartnett]

In general we can't depend on users to notice output to stdout. Many real-world programs generate lots of output, and any error messages will be lost in the noise. The standard C library approach is to return non-zero error codes.

The loss of parallel I/O might be noticed by attentive programmers but that is not something we should rely on. If they open a file for parallel I/O and it does not work, they should receive an unambiguous failure code. If they then choose to roll over to a sequential mode, that is of course their decision.

Basically to fail over to sequential mode violates the law of least surprise. As a programmer using C libraries I would very much expect that if I can't get what I ask for, I should get an error code and be able to decide what to do next.

The solution I outline does allow existing users to continue using existing createfile function if they want automatic roll-over to sequential mode. Or they can write the (quite trivial) code which can accomplish the same with the proposed PIOc_create() function.

This is how all the other mode parameters work with netCDF. For example if you open a file read/write, and that does not work, it does not open the file read-only and give you that. You get an error code, because what you asked for does not work. If your application then chooses to try to open the file read-only, that's fine. But in my experience, most users would expect to make that decision themselves.