Mersenne mind twister

[climbfuji]

Description

@DusanJovic-NOAA reported an issue with running inline post on a generic Linux machine with the GNU compilers. This was tracked down to a multitude of mersenne twister random generators being used across the components of the ufs-weather-model, some of them compiled in single precision, some in double precision.

Whether the correct one is used or not depends on the compiler and how the build system generates the linker command for that compiler. Intel apparently doesn't have the problem reported here.

Thus far we identified:

• one mersenne twister in w3emc
• two mersenne twisters in CCPP physics (either coming from IPD physics or being added as part of the HWRF physics)
• one mersenne twister in stochastic_physics
• one mersenne twister in FMS

The FMS version seems to be somewhat similar to the HWRF version, but according to @DusanJovic-NOAA the module in which it resides in FMS is different from the others and thus doesn't pose a problem right now.

Proposed solution - open for discussion

1. Identify if the different versions are really required or if one version, providing the interfaces for single and double precision (and potentially half / quadruple precision) is sufficient - after all, it is the same algorithm. Move this version/these versions into its own library (yet another NCEPLIB) and use it in all applications.

2. Potential interim solution for post: replace mersenne twister with Fortran number generator, maybe only for GNU to avoid downstream effects on existing users until 1. has been completed.

To Reproduce:

Compile the ufs-weather-model on a Linux laptop/workstation with the GNU compiler using 32bit dynamics and run a test with inline post turned on. Create verbose output during the make step (VERBOSE=1) and check the final linker line.

Related issues

NCAR/ccpp-physics#529

[climbfuji]

One version of the Mersenne twister will be deleted in #331.

[arunchawla-NOAA]

I have not seen any discussion on this topic. Is this still an issue that needs to be addressed ? Are there plans to tackle this at this time or should this be moved to the icebox for later considerations?

[climbfuji]

I have not seen any discussion on this topic. Is this still an issue that needs to be addressed ? Are there plans to tackle this at this time or should this be moved to the icebox for later considerations?

icebox I guess. It's still the same, nobody has made efforts to consolidate the versions and find a good way to use them across repositories.

[pjpegion]

I have plans to remove the one in stochastic physics later this year.

[BrianCurtis-NOAA]

The one for UPP looks like it was switched to use the Fortran RANDOM_NUMBER generator. Does FMS/Stochastic Phys/CCPP need a Mersenne twister or will the RANDOM_NUMBER do OK? I ask because I think it good to track these changes in upstream tickets and link here, but I wanted to know if there was a need or if the RANDOM_NUMBER generator in Fortran would suffice.

[climbfuji]

It's up to the scientists to weight in here. Some parts of the model, e.g. radiation, depend heavily on the random number generator, and you can't just swap them because they have characteristics that can really influence the model.

[BrianCurtis-NOAA]

Yes, I think they use different distribution types? If the scientists in each component can weigh in we can get the proper tickets created there, agreed. Thanks!

[SMoorthi-emc]

Please do not remove mersenne twister from the model. We have been using this in GFS over more than a decade. Note that UPP is not prognostic i.e. it does not interact with forecast. Moorthi

…

On Mon, Jul 25, 2022 at 4:13 PM Brian Curtis ***@***.***> wrote: Yes, I think they use different distribution types? If the scientists in each component can weigh in we can get the proper tickets created there, agreed. Thanks! — Reply to this email directly, view it on GitHub <#324 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALLVRYRCVXAFQMTEZD2O6CLVV3YNBANCNFSM4UTSKHBA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Dr. Shrinivas Moorthi Research Meteorologist Modeling and Data Assimilation Branch Environmental Modeling Center / National Centers for Environmental Prediction 5830 University Research Court - (W/NP23), College Park MD 20740 USA Tel: (301)683-3718 e-mail: ***@***.*** Phone: (301) 683-3718 Fax: (301) 683-3718

[pjpegion]

I am ok with removing mersenne twister from the stochastic physics repo, but it will take a bit of coding and testing to switch to a different random number generator. The CA part of the code is no longer using mersenne twister, I just have not had the time to update the spectral pattern generator.