UFS WM regression test namelist template updates

[junwang-noaa]

This EPIC includes updates/cleanup of the UFS WM global regression test namelist templates updates so that the templates can be used in the global workflow to seamless code updates.

1. The global regression test namelist templates will be consolidated and generalized to support standalone and coupled configurations.
2. A procedure of adding new namelist for feature test will be set up.
3. Additional changes may be required when the templated are adopted in global workflow.

[dpsarmie]

(Edit: Moved these comments to #1248)

The first iteration of the consolidated namelist is located here: https://github.com/dpsarmie/ufs-weather-model/tree/feature/NML_consolidate_ATM

The ATM tests that use the original cpld NML work with these changes. Further work will be done to integrate other coupled tests to this master namelist.

[dpsarmie]

There is an issue with some tests that use a combination of FV3 variables and the coupled parameter values.
This issue was encountered with control_p8 (as an example).

The consolidated namelist was set up in a way where the default values in default_vars.sh either fall under the export_fv3 or export_cpld call. Most of the coupled systems use both of these calls to generate their parameter values.

However, control_p8 did not. Control_p8 only calls export_fv3 and used coupled values that were (formally) located in cpld.control.nml.IN.

Possible solutions:

1. Modify control_p8 in ./tests/tests/ to add these coupled values or call export_cpld to populate those values
2. Add a new call in default_vars.sh in addition to export_fv3 and export_cpld to deal with this 'hybrid' parameter case.

Full list of affected tests (basically all *p8 tests):
control_p8
control_debug_p8
control_decomp_p8
atmaero_control_p8_rad
control_restart_noqr_p8
control_p8_rrtmgp
control_p8_lndp
control_p8_ugwpv1
control_2threads_p8
control_noqr_p8
control_p8_faster
control_restart_p8
control_p8_mynn
atmaero_control_p8
atmwav_control_noaero_p8
control_p8_atmlnd_sbs

[dpsarmie]

Latest version of https://github.com/dpsarmie/ufs-weather-model/tree/feature/NML_consolidate_ATM now includes a master template for all cpld tests. I will integrate all other global versions next.

[junwang-noaa]

@dpsarmie Can you list the "'hybrid' parameters? My understanding is that since this is FV3atm namelist, these fields should all go to export_fv3. But I'd like to see these parameters before we make decision. Thanks.
@DeniseWorthen FYI.

[dpsarmie]

Hi @junwang-noaa and @DeniseWorthen ,

I'll use control_p8 as an example.

Control_p8 only calls export_fv3 but also uses cpld_control.nml.IN so those variable values are loaded.
(For example, tau = 10. in default fv3 but tau = 0.0 in the coupled namelist)

I'm moving parameters that are present in cpld_control.nml.IN into the export_cpld call in default_vars.sh.
Other coupled tests (ex. cpld_debug_gfsv17) call export_fv3 and export_cpld, so that's working with the combined namelist template.

Other fv3 only tests (ex. control_c192) only call export_fv3 and use control.nml.IN, which is also ok with the combined template.

These other "hybrid" tests have a combination of both (using only export_fv3 and the coupled namelist), which causes an issue that can be solved via a couple of different methods, which I proposed above. If you all have more elegant solutions, let me know!

Complete (hopefully) list of parameters affected:
range_warn, tau,fast_tau_w_sec,rf_cutoff,n_split,dry_mass,beta

========================================================
Update 02/01/2024:
Talked with Jun and developed a path forward.

[dpsarmie]

New update (@junwang-noaa @DeniseWorthen):

About 50 rt's have been run and tested so far.

One reason for the delay is the following:
The combined nml has pointers to all possible variables. These env variables are populated for all of the cpld runs that call export_fv3 and export_cpld. Other runs that only export_fv3 might populate these variables depending on the test, but the ones that don't crash due to undefined env variables even if the test run doesn't use those variables.

I tested just populating all env variables in default_vars.sh, which worked for some runs but caused others to consistently (defined as 3+ tries over at least 2 different days) to timeout over the allotted time.

I'm now populating the combined nml with blank pointers for variables that aren't being used by some of the control runs. This way they are defined but won't cause any runtime issues and will be populated by the regression tests that do use those variables (when export_cpld is called).