[develop] Split EnsembleStat vx tasks into GenEnsProd and EnsembleStat #809
Conversation
… now done in GenEnsProd (to avoid duplication).
…e output of GenEnsProd (instead of EnsembleStat, which no longer generates the necessary outputs).
…e desired dependencies possible; remove unneeded variables; change vx WE2E test config files to be compatible with aforementioned changes.
…for which to perform vx user-specifiable.
…hich fields or group of fields on which to perform verification.
… field(s) are usually come first in vx.
…fields for those metatasks.
…er or not to run deterministic and ensemble verification, respectively.
… them to work properly with latest changes.
…lus config files (and in order for the final config files to have the correct values), change the way the variable ENSMEM_INDX is passed into METplus config files from an environment variable to a jinja variable.
1) Replace "mem${ensmem_indx}" with new variable "${ensmem_name}".
2) Remove DOT_ENSMEM_OR_NULL since ensmem_name can now be used instead.
3) Replace SLASH_ENSMEM_SUBDIR_OR_NULL with slash_ensmem_subdir_or_null.
4) Remove unneeded code from exregional_run_met_pb2nc_obs.sh.
gsketefian
changed the title
gsketefian
requested review from
mkavulich,
JeffBeck-NOAA and
RatkoVasic-NOAA
as code owners
…ctionality is replaced by inclusion/exclusion of workflow taskgroup yaml configuration files.
…_all_mems" from the workflow.
…of ensemble vx tasks in the workflow (by making sure that the names of the ensemble vx tasks are read in from the appropriate yaml file instead of being hard-coded).
…mble and deterministic forecasts) and the way the time lag is calculated for a given ensemble member (or the deterministic forecast).
|
@christinaholtNOAA I think I addressed all of your concerns except the renaming of the vx tasks (Issue #633). |
@gsketefian, thanks for all your work on this PR! I agree that we can tackle the renaming of the vx tasks in a separate PR after we consult with others. I'd personally be fine with just removing the "run" portion of the ex-script and j-job names for all vx tasks. |
…ation_only_vx_time_lag" to look for the staged forecast data from the default location for the machine; also, change the predefined grid name to the one on which this staged data is provided (this is necessary to form the correct path to the data).
…g any time-lagging) to names of GenEnsProd and EnsembleStat output files.
…GenEnsProd task, not the EnsembleStat task. Also, change order of metatasks so that the one for MRMS comes before the one for NDAS.
There was a problem hiding this comment.
@gsketefian These changes look good to me! I was also able to successfully run the new MET_ensemble_verification_only_vx_time_lag test, as well as the rest of the verification WE2E tests on Hera.
|
@gsketefian I will go ahead and submit the automated Jenkins testing on this work now. If @christinaholtNOAA requests additional changes, then the tests can be resubmitted at that time. |
MichaelLueken
added
the
run_we2e_coverage_tests
There was a problem hiding this comment.
@gsketefian Thanks for addressing my concerns! This looks great.
|
@gsketefian The automated Jenkins tests encountered failures: Cheyenne GNU - the Gaea - SRW won't build following the latest maintenance on the machine. Will be turning off Gaea testing in #799 until the UFS-WM has merged the new Gaea modulefile changes. Hera Intel - the I saw that the there were two failures in the Hera GNU test - I will also run the Jenkins tests manually on Orion, since the machine requires git/2.28.0 to be loaded, but since git/2.28.0 isn't available on Hercules and Hercules and Orion are the same machine, the git/2.28.0 module load is constantly removed from the role-epic-ps account's .bashrc file. I will let you know if there are any failures. |
|
@gsketefian Both the manual run of the Jenkins tests on Orion and the rerun of Hera Intel automated Jenkins tests have all successfully passed. I will now move forward with merging this work to develop. |
|
@MichaelLueken Thanks for running the tests and merging! |
* Remove RRFS_v1alpha suite
DESCRIPTION OF CHANGES:
The latest versions of MET/METplus have introduced a new tool named
GenEnsProdthat performs many of the ensemble-related calculations thatEnsembleStatused to do. AlthoughEnsembleStatcan still perform these calculations, that will change in future versions of MET/METplus. Thus, this PR creates new vx tasks that callGenEnsProdto perform the necessary calculations, removing them from theEnsembleStattasks. Details:GenEnsProd.exregional_run_met_genensprod_or_ensemblestat.shto enable it to callGenEnsProd(as well asEnsembleStat).EnsembleStatconfig files so thatEnsembleStatno longer calculates the ensemble statistics that are now the purview ofGenEnsProd.GenEnsProdtasks, not the output of theEnsembleStattasks.verify.yamlandverify_ensgrid.yamltoverify_det.yamlandverify_ens.yaml, respectively.verify_pre.yamlthat contains (meta)tasks that are prerequisites for both deterministic and ensemble vx (these tasks were previously included inverify_det.yaml).verify_ens.yamlto include the newGenEnsProdtasks.verify_det.yamlandverify_ens.yamlso that vx task dependencies can be formulated more precisely, e.g. have dependencies on member-specific tasks instead of whole metatasks. This includes splitting metatasks into two or more other ones, moving tasks from one metatask to another, etc.MET_verification,MET_verification_only_vx,MET_ensemble_verification, andMET_ensemble_verification_only_vx) to reflect the changes in the default vx workflow configuration filesverify_pre.yaml,verify_det.yaml, andverify_ens.yaml.VX_FIELDSto enable the user to specify the fields or groups of fields on which to perform verification.VX_APCP_ACCUMS_HHto enable the user to specify the accumulation hours for which to perform verification on accumulated precip (APCP).setup.pyto remove any [meta]tasks for which fields are not specified inVX_FIELDS.setup.pyso that for configuration parameters that are arrays/lists, each value of the list is checked to ensure it is valid (if that variable has a correspondingvalid_vals_array invalid_param_vals.yaml).config_defaults.shto ensure that the vx tests succeed in NCO mode.ENS_TIME_LAG_HRSis set and accessed correctly for both deterministic and ensemble configurations. For a deterministic configuration, this is a 1-element array containing the time lag (if any, in units of hours), while for an ensemble configuration it containsNUM_ENS_MEMBERSelements containing the time lag for each element.MET_ensemble_verification_only_vx_time_lag) that performs deterministic and ensemble verification on a 2-member ensemble with one time-lagged and one non-time-lagged member.MET_ensemble_verification_only_vxand the new testMET_ensemble_verification_only_vx_time_lagto the list of comprehensive WE2E tests.exregional_get_obs_[ccpa|mrms|ndas].shex-scripts.Type of change
TESTS CONDUCTED:
I ran the following vx WE2E tests on Hera with intel, both in community mode and in NCO mode:
I also ran the following
fundamentalsuite of tests:All tests were successful.
DOCUMENTATION:
Documentation is needed but will be added once the whole set of vx PRs has been merged (see Issue #630).
CHECKLIST