20220414 BoCui sync and merge UPP/develop into post_2d_decomp (#7)

* Added lambert conformal projection for FV3SAR and handled composite reflectivity correctly (NOAA-EMC#14) (#2) * exec is selectable for executable directory (NOAA-EMC#448) * Doxygen for CALGUST (NOAA-EMC#451) * Doxygen for CALDRG.f * This is part of Issue NOAA-EMC#392 Fixes the doxygen warnings in CALGUST.f Please review this subroutine. Thank you. * The fix in CALUPDHEL.f (NOAA-EMC#458) * Modify CALUPDHEL.f for restricting undefined grids in computation. * Update VERSION to 10.0.12. * Bug fix for SLLEVEL bound issue when not RUC LSM (NOAA-EMC#463) Co-authored-by: Tracy <[email protected]> * Doxygen caldrg caldwcp calgust refinement (NOAA-EMC#464) * Further refinement to the tables and logs. * Further refinement to the tables and logs. (NOAA-EMC#455) * This is part of Issue NOAA-EMC#392 (NOAA-EMC#460) Fixes the doxygen warnings in CALHEL.f CALHEL2.f CALHEL3.f Please review this subroutine. Thank you. * Unify global and regional FV3 read interfaces (NOAA-EMC#453) * Unify the interfaces for reading FV3 outputs in netcdf. * Remove interface INITPOST_GFS_NETCDF_PARA. * Remove INITPOST_GFS_NETCDF.f. * Remove the capability of serial netcdf reading FV3 outputs. * Correct reading rswinc. * Remove duplication in CLDRAD.f. * Remove duplicated avgalbedo reading * Add changes for reading pwat from model. * Clean up commented out code * Clean up duplicated lines * Add ability to compile script to use non-intel compilers; add Cheyenne modulefiles for gnu and intel (NOAA-EMC#468) * Add cheyenne modulefile * Intel 19 --> 2021 * Add ability to specify compiler; move all existing modulefiles to ${name}_intel since they are all for intel compilers; add "cheyenne_gnu" modulefile * Revert move of intel modulefiles; now the implicit default for a modulefile is intel, only gnu and other compilers will have the compiler name appended to the modulefile * Allow for lua modulefiles with ".lua" appended * This is part of Issue NOAA-EMC#392 (NOAA-EMC#465) Fixes the doxygen warnings in CALLCL.f CALMCVG.f CALMICT.f * This is part of Issue NOAA-EMC#392 (NOAA-EMC#466) Fixes the doxygen warnings in CALPBL.f CALPBLREGIME.f CALPOT.f CALPW.f. * Doxygen in CALRAD_WCLOUD_newcrtm.f CALRCH.f CALSTRM.f CALTAU.f CALTHTE.f CALUPDHEL.f. (NOAA-EMC#467) * This is part of Issue NOAA-EMC#392. Fixes the doxygen warnings in CALRAD_WCLOUD_newcrtm.f CALRCH.f CALSTRM.f CALTAU.f CALTHTE.f CALUPDHEL.f. * Fixed typo. * Update to PR NOAA-EMC#458. * Doxygen in CALVOR.f, CALWXT_BOURG.f, CLDRAD.f, COLLECT.f, COLLECT_LOC.f, DEALLOCATE.f, and DEALLOCATE.f (NOAA-EMC#469) * This is part of Issue NOAA-EMC#392. Fixes the doxygen warnings in CALVOR.f, CALWXT_BOURG.f, CLDRAD.f, COLLECT.f, COLLECT_LOC.f, DEALLOCATE.f, and DEALLOCATE.f. * Fixed minor bug in CALVOR.f. * Update to PR#453. * More updates. * 20220411 Bo Cui 2D decompositio CALVOR.f and INITPOST_NETCDF.f * 20220411 Bo Cui update doxgen and global and regional FV3 read interface * 20220414 Bo Cui: remove read_netcdf_2d_scatter and read_netcdf_3d_scatter from INITPOST_NETCDF.f * 20220415 Bo Cui delete CALVOR.f, add exch of gdlon in INITPOST_NETCDF.f * 20220421 Bo Cui Doxygen in UPP_PHYSICS.f,add restriction run 2D decomp only for GFS/FV3R in WRFPOST.f Co-authored-by: WenMeng-NOAA <[email protected]> Co-authored-by: Wen Meng <[email protected]> Co-authored-by: Chan-Hoo.Jeon-NOAA <[email protected]> Co-authored-by: kayeekayee <[email protected]> Co-authored-by: Tracy Hertneky <[email protected]> Co-authored-by: Tracy <[email protected]> Co-authored-by: Michael Kavulich <[email protected]>
WenMeng-NOAA · Apr 22, 2022 · 5186eb2 · 5186eb2
1 parent d3d5954
commit 5186eb2
Show file tree

Hide file tree

Showing 39 changed files with 2,107 additions and 8,090 deletions.
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -63,6 +63,12 @@ if(BUILD_POSTEXEC)
   endif()
 endif()
 
+### Switch RUNTIME DESTINATION DIR between bin and exec
+set(exec_dir bin)
+if(EMC_EXEC_DIR)
+  set(exec_dir exec)
+endif()
+
 add_subdirectory(sorc)
 add_subdirectory(parm)
 

diff --git a/README.md b/README.md
@@ -110,9 +110,9 @@ Builds include:
 ```
 mkdir build
 cd build
-cmake .. -DCMAKE_INSTALL_PREFIX=/path/to/install 
-make 
-make test
+cmake .. -DCMAKE_INSTALL_PREFIX=/path/to/install
+(or cmake .. -DCMAKE_INSTALL_PREFIX=/path/to/install -DEMC_EXEC_DIR=ON)
+make -j 4
 make install
 ```
 

diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-10.0.11
+10.0.12
diff --git a/modulefiles/cheyenne b/modulefiles/cheyenne
@@ -0,0 +1,40 @@
+#%Module#
+
+proc ModulesHelp { } {
+puts stderr "Loads modules required for building upp"
+}
+module-whatis "Loads UPP prerequisites on Cheyenne for Intel 2021.2"
+
+module purge
+
+module load cmake/3.18.2
+module load ncarenv/1.3
+module load intel/2021.2
+module load mpt/2.22
+module load ncarcompilers/0.5.0
+module unload netcdf
+
+module use /glade/p/ral/jntp/GMTB/tools/hpc-stack-v1.2.0/modulefiles/stack
+module load hpc/1.2.0
+module load hpc-intel/2021.2
+module load hpc-mpt/2.22
+
+module load jasper/2.0.25
+module load zlib/1.2.11
+module load png/1.6.35
+
+module load hdf5/1.10.6
+module load netcdf/4.7.4
+
+module load bacio/2.4.1
+module load crtm/2.3.0
+module load g2/3.4.2
+module load g2tmpl/1.10.0
+module load ip/3.3.3
+module load nemsio/2.5.2
+module load sfcio/1.4.1
+module load sigio/2.3.2
+module load sp/2.3.3
+module load w3nco/2.4.1
+module load w3emc/2.7.3
+module load wrf_io/1.2.0
diff --git a/modulefiles/cheyenne_gnu b/modulefiles/cheyenne_gnu
@@ -0,0 +1,41 @@
+#%Module#
+
+proc ModulesHelp { } {
+puts stderr "Loads modules required for building upp"
+}
+module-whatis "Loads UPP prerequisites on Cheyenne for Intel 2021.2"
+
+module purge
+
+module load cmake/3.18.2
+module load ncarenv/1.3
+module load gnu/10.1.0
+module load mpt/2.22
+module load ncarcompilers/0.5.0
+module load python/3.7.9
+module unload netcdf
+
+module use /glade/p/ral/jntp/GMTB/tools/hpc-stack-v1.2.0/modulefiles/stack
+module load hpc/1.2.0
+module load hpc-gnu/10.1.0
+module load hpc-mpt/2.22
+
+module load jasper/2.0.25
+module load zlib/1.2.11
+module load png/1.6.35
+
+module load hdf5/1.10.6
+module load netcdf/4.7.4
+
+module load bacio/2.4.1
+module load crtm/2.3.0
+module load g2/3.4.2
+module load g2tmpl/1.10.0
+module load ip/3.3.3
+module load nemsio/2.5.2
+module load sfcio/1.4.1
+module load sigio/2.3.2
+module load sp/2.3.3
+module load w3nco/2.4.1
+module load w3emc/2.7.3
+module load wrf_io/1.2.0
diff --git a/sorc/ncep_post.fd/ALLOCATE_ALL.f b/sorc/ncep_post.fd/ALLOCATE_ALL.f
@@ -18,6 +18,7 @@
 !! -  21-04-06  Wen Meng - Initializing all allocated arrays
 !! -  21-04-16  Wen Meng - Initializing aextc55 and extc55 as 0. These
 !!                      two arrays are involved in GSL visibility computation.
+!! -  22-03-22  Wen Meng - Initializing pwat.
 !!
 !!   OUTPUT FILES:
 !!   - STDOUT  - RUN TIME STANDARD OUT.
@@ -970,6 +971,7 @@ SUBROUTINE ALLOCATE_ALL()
       allocate(tedir(ista_2l:iend_2u,jsta_2l:jend_2u))
       allocate(twa(ista_2l:iend_2u,jsta_2l:jend_2u))
       allocate(fdnsst(ista_2l:iend_2u,jsta_2l:jend_2u))
+      allocate(pwat(ista_2l:iend_2u,jsta_2l:jend_2u))
 !Initialization
 !$omp parallel do private(i,j)
       do j=jsta_2l,jend_2u
@@ -1020,6 +1022,7 @@ SUBROUTINE ALLOCATE_ALL()
           tedir(i,j)=spval
           twa(i,j)=spval
           fdnsst(i,j)=spval
+          pwat(i,j)=spval
         enddo
       enddo
 !

diff --git a/sorc/ncep_post.fd/CALDRG.f b/sorc/ncep_post.fd/CALDRG.f
@@ -2,17 +2,20 @@
 !> @brief Subroutine that computes drag cofficient.
 !     
 !> This rountine computes a surface layer drag coefficient using
-!> equation (7.4.1A) in "An introduction to boundary layer
-!> meteorology" by Stull (1988, Kluwer Academic Publishers).
+!> equation (7.4.1A) in ["An introduction to boundary layer
+!> meteorology" by Stull (1988, Kluwer Academic
+!> Publishers)](https://link.springer.com/book/10.1007/978-94-009-3027-8).
 !>     
-!> @param[out] DRAGCO surface layer drag coefficient
+!> @param[out] DRAGCO surface layer drag coefficient.
 !>
-!> Program history
-!> - 93-09-01  Russ Treadon
-!> - 98-06-15  T Black - Conversion from 1-D to 2-D
-!> - 00-01-04  Jim Tuccillo - MPI version           
-!> - 02-01-15  Mike Baldwin - WRF version
-!> - 05-02-22  H Chuang - Add WRF NMM components 
+!> ### Program History Log
+!> Date | Programmer | Comments
+!> -----|------------|---------
+!> 1993-09-01 | Russ Treadon | Initial
+!> 1998-06-15 | T Black      | Conversion from 1-D to 2-D
+!> 2000-01-04 | Jim Tuccillo | MPI version           
+!> 2002-01-15 | Mike Baldwin | WRF version
+!> 2005-02-22 | H Chuang     | Add WRF NMM components 
 !>
 !> @author Russ Treadon W/NP2 @date 1993-09-01
       SUBROUTINE CALDRG(DRAGCO)

diff --git a/sorc/ncep_post.fd/CALDWP.f b/sorc/ncep_post.fd/CALDWP.f
@@ -1,21 +1,21 @@
 !> @file
 !> @brief Subroutine related to dewpoint temperature.
 !
-!> Computes dewpoint from P, T, and Q
+!> Computes dewpoint from P, T, and Q.
 !>     
-!> @param[in] P1D Pressure (Pa)
-!> @param[in] Q1D Specific humidity (kg/kg)
-!> @param[in] T1D Temperature (K)
-!> @param[out] TDWP Dewpoint temperature (K)
+!> @param[in] P1D Pressure (Pa).
+!> @param[in] Q1D Specific humidity (kg/kg).
+!> @param[in] T1D Temperature (K).
+!> @param[out] TDWP Dewpoint temperature (K).
 !>
-!> Program history
-!> - 92-12-22  Russ Treadon
-!> - 93-10-04  Russ Treadon - Added check to bound dewpoint
-!>                            temperature to not exceed the
-!>                            ambient temperature.
-!> - 98-06-08  T BLACK      - Conversion from 1-D to 2-D
-!> - 00-01-04  Jim Tuccillo - MPI version                
-!> - 21-07-23  Wen Meng     - Retrict computation from undefined points
+!> ### Program history log:
+!> Date | Programmer | Comments
+!> -----|------------|---------
+!> 1992-12-22 | Russ Treadon | Initial
+!> 1993-10-04 | Russ Treadon | Added check to bound dewpoint temperature to not exceed the ambient temperature.
+!> 1998-06-08 | T Black      | Conversion from 1-D to 2-D
+!> 2000-01-04 | Jim Tuccillo | MPI version                
+!> 2021-07-23 | Wen Meng     | Retrict computation from undefined points
 !>     
 !> @author Russ Treadon W/NP2 @date 1992-12-22
       SUBROUTINE CALDWP(P1D,Q1D,TDWP,T1D)

diff --git a/sorc/ncep_post.fd/CALGUST.f b/sorc/ncep_post.fd/CALGUST.f
@@ -1,47 +1,22 @@
 !> @file
-!                .      .    .     
-!> SUBPROGRAM:    CALGUST      COMPUTE MAX WIND LEVEL 
-!!   PRGRMMR: MANIKIN        ORG: W/NP2   DATE: 97-03-04       
-!!     
-!! ABSTRACT:  
-!!     THIS ROUTINE COMPUTES SURFACE WIND GUST BY MIXING
-!!  DOWN MOMENTUM FROM THE LEVEL AT THE HEIGHT OF THE PBL
-!!     
-!!     
-!! PROGRAM HISTORY LOG:
-!!   03-10-15 GEOFF MANIKIN
-!!   05-03-09 H CHUANG - WRF VERSION
-!!   05-07-07 BINBIN ZHOU - ADD RSM   
-!!   15-03-11 S Moorthi - set sfcwind to spval if u10 and v10 are spvals
-!!                        for A grid and set gust to just wind
-!!                        (in GSM with nemsio, it appears u10 & v10 have spval)
-!!   21-09-02  Bo Cui   - Decompose UPP in X direction
-!!   
-!! USAGE:    CALL CALGUST(GUST) 
-!!   INPUT ARGUMENT LIST:
-!!     NONE     
-!!
-!!   OUTPUT ARGUMENT LIST: 
-!!     GUST    - SPEED OF THE MAXIMUM SFC WIND GUST 
-!!     
-!!   OUTPUT FILES:
-!!     NONE
-!!     
-!!   SUBPROGRAMS CALLED:
-!!     UTILITIES:
-!!       H2V     
-!!
-!!     LIBRARY:
-!!       COMMON   - 
-!!                  LOOPS
-!!                  OPTIONS
-!!                  MASKS
-!!                  INDX
-!!     
-!!   ATTRIBUTES:
-!!     LANGUAGE: FORTRAN 90
-!!     MACHINE : CRAY C-90
-!!
+!> @brief Subroutine that computes max wind level.
+!
+!> This routine computes surface wind gust by mixing
+!> down momentum from the level at the height of the PBL.
+!>     
+!> @param[out] GUST Speed of the maximum surface wind gust.
+!>
+!> ### Program history log:
+!> Date | Programmer | Comments
+!> -----|------------|---------
+!> 2003-10-15 | Geoff Manokin | Initial
+!> 2005-03-09 | H Chuang      | WRF Version
+!> 2005-07-07 | Binbin Zhou   | Add RSM   
+!> 2015-03-11 | S Moorthi     | Set sfcwind to spval if u10 and v10 are spvals for A grid and set gust to just wind (in GSM with nemsio, it appears u10 & v10 have spval)
+!> 2021-09-02 | Bo Cui        | Decompose UPP in X direction
+!>   
+!> @author Geoff Manikin W/NP2 @date 1997-03-04
+
       SUBROUTINE CALGUST(LPBL,ZPBL,GUST)
 
 !     

diff --git a/sorc/ncep_post.fd/CALHEL.f b/sorc/ncep_post.fd/CALHEL.f
@@ -1,81 +1,44 @@
 !> @file
-!                .      .    .     
-!> SUBPROGRAM:    CALHEL       COMPUTES STORM RELATIVE HELICITY
-!!   PRGRMMR: BALDWIN         ORG: W/NP2      DATE: 94-08-22       
-!!     
-!! ABSTRACT:
-!!     THIS ROUTINE COMPUTES ESTIMATED STORM MOTION AND
-!!     STORM-RELATIVE ENVIRONMENTAL HELICITY.  
-!!     (DAVIES-JONES ET AL 1990) THE ALGORITHM PROCEEDS AS 
-!!     FOLLOWS.
-!!     
-!!     THE STORM MOTION COMPUTATION NO LONGER EMPLOYS THE DAVIES AND
-!!     JOHNS (1993) METHOD WHICH DEFINED STORM MOTION AS 30 DEGREES TO
-!!     THE RIGHT OF THE 0-6 KM MEAN WIND AT 75% OF THE SPEED FOR MEAN
-!!     SPEEDS LESS THAN 15 M/S AND 20 DEGREES TO THE RIGHT FOR SPEEDS
-!!     GREATER THAN 15 M/S.   INSTEAD, WE NOW USE THE DYNAMIC METHOD
-!!     (BUNKERS ET AL. 1998) WHICH HAS BEEN FOUND TO DO BETTER IN
-!!     CASES WITH 'NON-CLASSIC' HODOGRAPHS (SUCH AS NORTHWEST-FLOW
-!!     EVENTS) AND DO AS WELL OR BETTER THAN THE OLD METHOD IN MORE
-!!     CLASSIC SITUATIONS. 
-!!     
-!! PROGRAM HISTORY LOG:
-!!   94-08-22  MICHAEL BALDWIN
-!!   97-03-27  MICHAEL BALDWIN - SPEED UP CODE
-!!   98-06-15  T BLACK         - CONVERSION FROM 1-D TO 2-D
-!!   00-01-04  JIM TUCCILLO    - MPI VERSION
-!!   00-01-10  G MANIKIN       - CHANGED TO BUNKERS METHOD
-!!   02-05-22  G MANIKIN       - NOW ALLOW CHOICE OF COMPUTING
-!!                               HELICITY OVER TWO DIFFERENT
-!!                               (0-1 and 0-3 KM) DEPTHS
-!!   03-03-25  G MANIKIN       - MODIFIED CODE TO COMPUTE MEAN WINDS
-!!                               USING ARITHMETIC AVERAGES INSTEAD OF
-!!                               MASS WEIGHTING;  DIFFERENCES ARE MINOR
-!!                               BUT WANT TO BE CONSISTENT WITH THE
-!!                               BUNKERS METHOD
-!!   04-04-16  M PYLE          - MINIMAL MODIFICATIONS, BUT PUT INTO
-!!                                NMM WRFPOST CODE
-!!   05=02-25  H CHUANG        - ADD COMPUTATION FOR ARW A GRID
-!!   05-07-07  BINBIN ZHOU     - ADD RSM FOR A GRID  
-!!   19-10-30  Bo CUI          - REMOVE "GOTO" STATEMENT
-!!   21-09-02  Bo Cui       - Decompose UPP in X direction
-!!   
-!! USAGE:    CALHEL(UST,VST,HELI)
-!!   INPUT ARGUMENT LIST:
-!!     DPTH      - DEPTH IN METERS OVER WHICH HELICITY SHOULD BE COMPUTED;
-!!                 ALLOWS ONE TO DISTINGUISH 0-3 KM AND 0-1 KM VALUES
-!!
-!!   OUTPUT ARGUMENT LIST: 
-!!     UST      - ESTIMATED U COMPONENT (M/S) OF STORM MOTION.
-!!     VST      - ESTIMATED V COMPONENT (M/S) OF STORM MOTION.
-!!     HELI     - STORM-RELATIVE HELICITY (M**2/S**2)
-!! CRA
-!!     USHR1    - U COMPONENT (M/S) OF 0-1 KM SHEAR
-!!     VSHR1    - V COMPONENT (M/S) OF 0-1 KM SHEAR
-!!     USHR6    - U COMPONENT (M/S) OF 0-0.5 to 5.5-6.0 KM SHEAR
-!!     VSHR6    - V COMPONENT (M/S) OF 0-0.5 to 5.5-6.0 KM SHEAR
-!! CRA
-
-!!     
-!!   OUTPUT FILES:
-!!     NONE
-!!     
-!!   SUBPROGRAMS CALLED:
-!!     UTILITIES:
-!!
-!!     LIBRARY:
-!!       COMMON   - VRBLS
-!!                  LOOPS
-!!                  PHYS 
-!!                  EXTRA
-!!                  MASKS
-!!                  OPTIONS
-!!                  INDX
-!!     
-!!   ATTRIBUTES:
-!!     LANGUAGE: FORTRAN 90
-!!     MACHINE : IBM SP
-!!
+!> @brief Subroutine that computes storm relative helicity.
+!
+!> This routine computes estimated storm motion and storm-relative
+!> environmental helicity. (Davies-Jones et al 1990) the algorithm
+!> processd as follows.
+!>     
+!> The storm motion computation no longer employs the Davies and Johns (1993)
+!> method which defined storm motion as 30 degress to the right of the 0-6 km
+!> mean wind at 75% of the speed for mean speeds less than 15 m/s and 20 degrees
+!> to the right for speeds greater than 15 m/s. Instead, we now use the dynamic
+!> method (Bunkers et al. 1988) which has been found to do better in cases with
+!> 'non-classic' hodographs (such as Northwest-flow events) and do as well or 
+!> better than the old method in more classic situations.
+!>     
+!> @param[in] DPTH Depth in meters over whcih helicity should be computed; allows one to distinguish 0-3 km and 0-1 km values.
+!> @param[out] UST Estimated U Component (m/s) Of Storm motion.
+!> @param[out] VST Estimated V Component (m/s) Of Storm motion.
+!> @param[out] HELI Storm-relative heliciry (m**2/s**2).
+!> @param[out] USHR1 U Component (m/s) Of 0-1 km shear.
+!> @param[out] VSHR1 V Component (m/s) Of 0-1 km shear.
+!> @param[out] USHR6 U Component (m/s) Of 0-0.5 to 5.5-6.0 km shear.
+!> @param[out] VSHR6 V Component (m/s) Of 0-0.5 to 5.5-6.0 km shear.
+!>     
+!> ### Program history log:
+!> Date | Programmer | Comments
+!> -----|------------|---------
+!> 1994-08-22 | Michael Baldwin | Initial
+!> 1997-03-27 | Michael Baldwin | Speed up code
+!> 1998-06-15 | T Black         | Conversion from 1-D to 2-D
+!> 2000-01-04 | Jim Tuccillo    | MPI Version
+!> 2000-01-10 | G Manikin       | Changed to Bunkers method
+!> 2002-05-22 | G Manikin       | Now allow choice of computing helicity over two different (0-1 and 0-3 km) depths
+!> 2003-03-25 | G Manikin       | Modified code to compute mean winds using arithmetic averages instead of mass weighting; differences are minor but want to be consistent with the Bunkers method
+!> 2004-04-16 | M Pyle          | Minimal modifications but put into NMM WRFPOST code
+!> 2005-02-25 | H Chuang        | Add computation for ARW A grid
+!> 2005-07-07 | Binbin Zhou     | Add RSM for A grid  
+!> 2019-10-30 | Bo Cui          | Remove "goto" statement
+!> 2021-09-02 | Bo Cui          | Decompose UPP in X direction          
+!>   
+!> @author Michael Baldwin W/NP2 @date 1994-08-22
       SUBROUTINE CALHEL(DEPTH,UST,VST,HELI,USHR1,VSHR1,USHR6,VSHR6)
 
 !