NaN values of attributes get converted to 0.0 when passed through opendap

[samoylovfp]

I have a .nc file with a variable with an attribute missing_value = NaNf. When I download this file from opendap in netcdf3 or netcdf4 format I get missing_value = 0.0f in the resulting file.

I expect to get the same NaNf as in the original file

Here's a script to reproduce the issue (requires bash, docker, wget and ncdump)
https://gist.github.com/samoylovfp/ba1df8ef41f27c44ddeed9f2d4076fc9

I suspect that istringstream is the culprit in these places

https://github.com/OPENDAP/bes/blob/6724e44/modules/fileout_netcdf/FONcAttributes.cc#L389
https://github.com/OPENDAP/bes/blob/6724e44/modules/fileout_netcdf/FONcAttributes.cc#L1030

Because it cannot parse NaN:

https://ideone.com/bhTntI
https://stackoverflow.com/questions/11420263/is-it-possible-to-read-infinity-or-nan-values-using-input-streams

[ndp-opendap]

Can you provide a link to the original file?

[samoylovfp]

Here's a link to a file with attributes being NaN
https://github.com/samoylovfp/binaries/blob/main/with_nan.nc

[jgallagher59701]

On Oct 5, 2020, at 8:52 AM, samoylovfp ***@***.***> wrote: I have a .nc file with a variable with an attribute missing_value = NaNf. When I download this file from opendap in netcdf3 or netcdf4 format I get missing_value = 0.0f in the resulting file. Here's a script to reproduce the issue https://gist.github.com/samoylovfp/ba1df8ef41f27c44ddeed9f2d4076fc9 <https://gist.github.com/samoylovfp/ba1df8ef41f27c44ddeed9f2d4076fc9> I suspect that istringstream is the culprit in these places https://github.com/OPENDAP/bes/blob/6724e44/modules/fileout_netcdf/FONcAttributes.cc#L389 <https://github.com/OPENDAP/bes/blob/6724e44/modules/fileout_netcdf/FONcAttributes.cc#L389> https://github.com/OPENDAP/bes/blob/6724e44/modules/fileout_netcdf/FONcAttributes.cc#L1030 <https://github.com/OPENDAP/bes/blob/6724e44/modules/fileout_netcdf/FONcAttributes.cc#L1030> Because it cannot parse NaN: https://ideone.com/bhTntI <https://ideone.com/bhTntI> https://stackoverflow.com/questions/11420263/is-it-possible-to-read-infinity-or-nan-values-using-input-streams <https://stackoverflow.com/questions/11420263/is-it-possible-to-read-infinity-or-nan-values-using-input-streams>

We’ll work this into the server ASAP, but this week we’re pretty well booked up. Thanks for the report. James

…

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#420>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7Q4KSVGPUGCM73OJESQ3DSJHMSPANCNFSM4SEZM5CA>.

-- James Gallagher [email protected]

[samoylovfp]

Is there anything I can do to help get this done?

[samoylovfp]

Hello!
The reply message said that this will be worked on ASAP but there is no update for over a month.
Just checking that this didn't fall through cracks.

I've tried to fix this on my own but I failed to correctly build the BES service, I had a number of problems with finding correct versions of dependency libraries. Is it possible to share the hyrax-dependencies-build.tar.gz and libdap-build.tar.gz that are used in the travis job?

[ndp-opendap]

@samoylovfp We are working on this issue in our sprint beginning today.

If you are still wishing to build the server you might want to review this set of instructions:

https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build

Which will explain how to deal with the hyrax-dependencies.

[sawer297]

@samoylovfp
@ndp-opendap and I found the issue within the code causing the issue.
So we will be implementing a fix for this, which should be complete by the end of next week.
Sorry for the delay but we are experiencing some unwanted throttling in our software testing process which is slowing the releasing of code fixes.

[larstiq]

@sawer297 thanks! If you have a branch we can build from we could hopefully do some testing with our data as well.

[sawer297]

@larstiq @samoylovfp
the fix has been applied, new regression tests added, and has been merged to the master branch
go ahead and pull the master branch and test using that
let us know if anymore bugs pop up

[samoylovfp]

The issue is fixed in Hyrax 1.16.3, thanks a lot!