WCOSS2 module and build updates #444
Conversation
- add if-block for NOAA Cactus and Dogwood to both module-setup.csh.inc and module-setup.sh.inc - source modules, purge modules, then load needed envvar/1.0 module to unlock library stacks Refs: NOAA-EMC#399
- add if-block for NOAA Cactus and Dogwood - initialize modules, purge modules, load needed envvar/1.0 module to unlock library stacks Refs: NOAA-EMC#399
- add WCOSS2 modulefiles for OznMonBuild and RadmonBuild - add WCOSS2 modulefile for storm_reloc - add WCOSS2 modulefiles for workflow util codes: enkf_chgres_recenter, enkf_chgres_recenter_nc, gaussian_sfcanl, gfs_bufr gfs_fbwndgfs, fv3nc2nemsio, regrid_nemsio, gfs_util, mkgfsawps, rdbfmsua Refs: NOAA-EMC#399
- the production library installation on WCOSS2 is hpc-stack and includes updates to some library variable names - added an if-block to build_gfs_bufrsnd.sh when on WCOSS2 to set the SIGIO variables to the new hpc-stack versions - SIGIO_LIB4 in hpc-stack is now just SIGIO_LIB - SIGIO_INC4 in hpc-stack is now just SIGIO_INC - need if-block to retain support for pre-hpc-stack in operations branch on other platforms Refs: NOAA-EMC#399
- the production library installation on WCOSS2 is hpc-stack and includes updates to some library variable names - added an if-block to build_tropcy_NEMS.sh when on WCOSS2 to set the SIGIO variables to the new hpc-stack versions - SIGIO_LIB4 in hpc-stack is now just SIGIO_LIB - SIGIO_INC4 in hpc-stack is now just SIGIO_INC - need if-block to retain support for pre-hpc-stack in operations branch on other platforms Refs: NOAA-EMC#399
- add CMD to end of exec copy to resolve copy error on WCOSS2 Refs: NOAA-EMC#399
- add WCOSS2 machine option to compile scripts for mkgfsawps, overgridid, rdbfmsua, and webtitle codes Refs: NOAA-EMC#399
- the parallel hdf5 and netcdf modules were renamed to match hpc-stack naming conventions: hdf5/1.10.6 and netcdf/4.7.4 - update module names in needed modulefiles Refs: NOAA-EMC#399
- create module_base modulefile for WCOSS2 - remaining modules to find are commented out: pm5, util_shared, HPSS, and cdo Refs: NOAA-EMC#399
- cleanup/compress the commands in build_fv3.sh for building the ufs-weather-model/FV3 code - based on recommended commands provided by Dusan Jovic Refs: NOAA-EMC#399
- add "wcoss2" to link script machine checks - add FIX_DIR path for temporary WCOSS2 fix_nco_gfsv16 set; will update to official glopara group location when available Refs: NOAA-EMC#399
- set checkout script to clone test copies for ufs-weather-model, GLDAS, and UPP - tested builds successfully; temporarily updated upp module in UPP build to be non-official copy from Wen Meng space (upp/8.0.11) Refs: NOAA-EMC#399
- there will be no HPSS module on WCOSS2 - hsi and htar are available in the environment Refs: NOAA-EMC#399
KateFriedman-NOAA
added
production update
KateFriedman-NOAA
requested review from
aerorahul and
WalterKolczynski-NOAA
There was a problem hiding this comment.
The OznMonBuild.wcoss2 and RadMonBuild.wcoss2 files can be deleted from the global_workflow. All of the monitoring components are within the GSI and enter the workflow through the DA checkout these days. My apologies for not noticing this earlier. I'll take a look through the global_workflow to see if there are any more legacy *Mon files that can be removed.
Thanks @EdwardSafford-NOAA ! I'll open a separate issue to remove those legacy files from the develop branch and will ping you in it for your inputs on that cleanup. Regarding removing those legacy files in the operations branch port to WCOSS2....@aerorahul @arunchawla-NOAA are we able to delete files in the transition to WCOSS2 or should we leave things as is if it doesn't need an update for WCOSS2? Thanks! |
If they are not going to be used for building these monitoring utilities, then just don't add them in this PR. |
- envvar/1.0 module will be loaded by module_base.wcoss2 - removing envvar/1.0 module load from module-setup.*.inc scripts Refs: NOAA-EMC#399
- undo prior commit - add envvar/1.0 back to module-setup.*.inc - remove envvar/1.0 from module_base.wcoss2 Refs: NOAA-EMC#399
- the OznMonBuild.wcoss2 and RadMonBuild.wcoss2 are not needed and are considered legacy since the GSI now handles OznMon/Radmon internally Refs: NOAA-EMC#399
- removed Hera from the check of WCOSS partitions in the util source code compile scripts Refs: NOAA-EMC#399
- NCO installed util_shared/1.4.0 module on WCOSS2 - need it in module_base.wcoss2 Refs: NOAA-EMC#399
- reorganize module load order to meet dependencies and order them based on groups: compilers, mpi, 3rd party, hdf5, netcdf, nceplibs - add modules to satisfy prereqs for current modules Refs: NOAA-EMC#399
- remove g2/3.4.1 module from gfs_util.wcoss2; not needed - reorganize modules in gfs_util.wcoss2 and add nemsio and sigio modules for w3emc prereqs - fix rdbfmsua makefile for successful compilation (Boi V provided) Refs: NOAA-EMC#399
- g2/3.4.1 reportedly degrades performance on WCOSS2 (Boi V) - update g2 module in storm_reloc module file from g2/3.4.1 to g2/3.4.5 Refs: NOAA-EMC#399
|
@aerorahul Please recheck the modulefiles, I've reorganized them. Let me know if any are still in the wrong order, thanks! @BoiVuong-NOAA I committed discussed updates to the util codes, please check I've made the updates correctly, thanks! |
- the nco module depends on netcdf, move it after netcdf module load Refs: NOAA-EMC#399
- add missing wgrib2 module load - update grib_util module version to 1.2.3 and add libjpeg/9c module load for dependency - add perl/5.32.0 module for Perl scripts, module was called pm5 on WCOSS-Dell Refs: NOAA-EMC#399
|
Tested fresh clone and build_all of branch on Cactus, no issues, although I forgot to add WCOSS2 to the machine check for the gfs_util build in build_all.sh. Committed that @ ea5357a. Good to submit this. |
|
@BoiVuong-NOAA Please sync merge your fork copy of feature/ops-wcoss2 with the updated auth repo copy now. You should be able to use the "Fetch upstream" button. Let me know via email or Slack if you run into issues. Thanks! |
|
Thank You, Kate. |
* Add capability for GNU and Add support for Cheyenne Co-authored-by: climbfuji <[email protected]>
In this PR are the following updates for WCOSS2 (being committed to the auth repo feature/ops-wcoss2 branch). This is a courtesy PR for comments ahead of submitting my fork branch changes into the auth repo branch. The full PR will happen when feature/ops-wcoss2 is merged into the operations branch. I will submit this PR if there are no objections or issues left to resolve next Tuesday. Thanks!
See a comment in related issue (#399 (comment)) for more details on these incoming changes to feature/ops-wcoss2 branch. No major changes will happen in this feature branch, only minimal updates to port to WCOSS2.
@EdwardSafford-NOAA please see the new WCOSS2 OznMon and RadMon build modulefiles.
The next PR will likely focus on the setup script updates. Rocoto will be installed on WCOSS2 soon so I can test the setup script produced xml when it's ready for use.