brings in FV3 "the dycore" updated to be consistent with the latest public release #772

bensonr · 2021-08-26T00:38:03Z

PR Checklist

Ths PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.
This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR
An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
are specified below.
If new or updated input data is required by this PR, it is clearly stated in the text of the PR.

Instructions: All subsequent sections of text should be filled in as appropriate.

The information provided below allows the code managers to understand the changes relevant to this PR, whether those changes are in the ufs-weather-model repository or in a subcomponent repository. Ufs-weather-model code managers will use the information provided to add any applicable labels, assign reviewers and place it in the Commit Queue. Once the PR is in the Commit Queue, it is the PR owner's responsiblity to keep the PR up-to-date with the develop branch of ufs-weather-model.

Description

This PR brings in an update for fv3atm that contains updates to the latest public release of the FV3 dycore encapsulated in PR #128.

This PR also disables the control_csawmg test for gaea.intel due to a numerical instability brought about by the updates to the dycore. The same test runs to completion on hera.intel.

This PR requires a new datestamp for the RTS ICs. The particular updates are to the gdas tests which pass regression test for this update.

This PR changes answers for all regional tests and any test with ICs created by chgres from sources that sets the data_source_fv3gfs=.true. option in FV3/atmos_cubed_sphere/tools/external_ic.F90::get_nggps_ic routine. Therefore new baselines will need to be created.

Issue(s) addressed

Link the issues to be closed with this PR, whether in this repository, or in another repository.
(Remember, issues must always be created before starting work on a PR branch!)

Testing

How were these changes tested? What compilers / HPCs was it tested with? Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) Have regression tests and unit tests (utests) been run? On which platforms and with which compilers? (Note that unit tests can only be run on tier-1 platforms)

Dependencies

If testing this branch requires non-default branches in other repositories, list them. Those branches should have matching names (ideally).

bensonr/GFDL_atmos_cubed_sphere/tree/master2emc
bensonr/fv3atm/tree/dycore

Do PRs in upstream repositories need to be merged first?
If so add the "waiting for other repos" label and list the upstream PRs

* bypass control_csawmg test for gaea (numerical instability)

junwang-noaa · 2021-08-26T12:24:16Z

Rusty identified following dycore code updates will cause baselines to change:

diff --git a/model/fv_regional_bc.F90 b/model/fv_regional_bc.F90
index 08f7c61..0ed9845 100644
--- a/model/fv_regional_bc.F90
+++ b/model/fv_regional_bc.F90
@@ -1271,7 +1270,0 @@ contains
-
-      do j=jsd,jed
-      do i=isd,ied
-        ps0(i,j) = Atm%ps(i,j)
-      enddo
-      enddo
-
@@ -1299,6 +1291,0 @@ contains
-
-      do j=jsd,jed
-      do i=isd,ied
-        Atm%ps(i,j) = ps0(i,j)
-      enddo
-      enddo
@@ -1417 +1404 @@ contains
-      ak_in(1) = max(1.e-9, ak_in(1))
+      ak_in(1) = 1.e-9
@@ -3401 +3388 @@ contains
-      allocate(BC_side%divgd_BC(is_we:ie_we,js_sn:je_sn,klev)) ; BC_side%divgd_BC=real_snan
+      allocate(BC_side%divgd_BC(is_0:ie_0,js_0:je_0,klev)) ; BC_side%divgd_BC=real_snan
@@ -4403 +4390 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v')then
@@ -4412 +4399 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v')then
@@ -4435 +4422 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v')then
@@ -4441 +4428 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc')then
@@ -4448 +4435 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v')then
@@ -4452 +4439 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc')then
@@ -4476 +4463 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='u')then
@@ -4489 +4476 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-              if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='divgd')then
+              if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc')then
@@ -4504 +4491 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc')then
@@ -4527 +4514 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='u')then
@@ -4534 +4521 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='uc'.or.trim(bc_vbl_name)=='v')then
@@ -4544 +4531 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-              if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='divgd')then
+              if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc')then
@@ -4559 +4546 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc'.or.trim(bc_vbl_name)=='divgd')then
+            if(trim(bc_vbl_name)=='u'.or.trim(bc_vbl_name)=='vc')then
@@ -5040,0 +5028 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
+              bc_side_t0%divgd_BC(i,j,k)=bc_side_t1%divgd_BC(i,j,k)
@@ -5080,6 +5068 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
-
-            do j=js_s,je_s
-            do i=is_w,ie_w
-              bc_side_t0%divgd_BC(i,j,k)=bc_side_t1%divgd_BC(i,j,k)
-            enddo
-            enddo
+          enddo
@@ -5086,0 +5070 @@ subroutine remap_scalar_nggps_regional_bc(Atm                         &
+          do k=1,nlev
diff --git a/tools/external_ic.F90 b/tools/external_ic.F90
index 443037c..a28e97e 100644
--- a/tools/external_ic.F90
+++ b/tools/external_ic.F90
@@ -798,5 +798,5 @@ contains
-            qt = wt/(1. - (Atm%q(i,j,k,liq_wat) + &
-                           Atm%q(i,j,k,ice_wat) + &
-                           Atm%q(i,j,k,rainwat) + &
-                           Atm%q(i,j,k,snowwat) + &
-                           Atm%q(i,j,k,graupel)))
+            qt = wt*(1. + Atm%q(i,j,k,liq_wat) + &
+                          Atm%q(i,j,k,ice_wat) + &
+                          Atm%q(i,j,k,rainwat) + &
+                          Atm%q(i,j,k,snowwat) + &
+                          Atm%q(i,j,k,graupel))
@@ -804 +804 @@ contains
-            qt = wt/(1. - sum(Atm%q(i,j,k,2:Atm%flagstruct%nwat)))
+            qt = wt*(1. + sum(Atm%q(i,j,k,2:Atm%flagstruct%nwat)))

bensonr · 2021-08-26T13:25:43Z

@junwang-noaa - thanks for including the diff stats. Please be aware the new/modified code is denoted by a '-' and the original code is the '+' in the diff listed above.

BrianCurtis-NOAA · 2021-08-26T14:23:17Z

Machine: cheyenne
Compiler: gnu
Job: BL
Repo location: /glade/scratch/dtcufsrt/autort/tests/auto/pr/720092158/20210826073009/ufs-weather-model
Please manually delete: /glade/scratch/dtcufsrt/FV3_RT/rt_10056
Baseline creation and move successful
Repo location: /glade/scratch/dtcufsrt/autort/tests/auto/pr/720092158/20210826075727/ufs-weather-model
Please manually delete: /glade/scratch/dtcufsrt/FV3_RT/rt_45380
Test fv3_esg_HAFS_v0_hwrf_thompson 015 failed failed
Test fv3_esg_HAFS_v0_hwrf_thompson 015 failed in run_test failed
Test control_ugwpv1 009 failed failed
Test control_ugwpv1 009 failed in run_test failed
Test control_thompson 007 failed failed
Test control_thompson 007 failed in run_test failed
Test control 001 failed failed
Test control 001 failed in run_test failed
Test control_thompson_no_aero 008 failed failed
Test control_thompson_no_aero 008 failed in run_test failed
Test fv3_gsd 011 failed failed
Test fv3_gsd 011 failed in run_test failed
Test control_stochy 004 failed failed
Test control_rrtmgp 006 failed failed
Test control_stochy 004 failed in run_test failed
Test control_rrtmgp 006 failed in run_test failed
Test control_flake 005 failed failed
Test control_flake 005 failed in run_test failed
Test control_ras 010 failed failed
Test control_ras 010 failed in run_test failed
Test control_ras_debug 028 failed failed
Test control_ras_debug 028 failed in run_test failed
Test control_noahmp_debug 029 failed failed
Test control_noahmp_debug 029 failed in run_test failed
Test control_debug 016 failed failed
Test control_debug 016 failed in run_test failed
Test control_diag_debug 017 failed failed
Test control_diag_debug 017 failed in run_test failed
Test fv3_gsd_debug 021 failed failed
Test fv3_gsd_debug 021 failed in run_test failed
Test control_c48 003 failed failed
Test control_c48 003 failed in run_test failed
Test fv3_rrfs_v1beta_debug 020 failed failed
Test fv3_rrfs_v1beta_debug 020 failed in run_test failed
Test control_ugwpv1_debug 027 failed failed
Test control_ugwpv1_debug 027 failed in run_test failed
Test control_thompson_extdiag_debug 025 failed failed
Test control_thompson_extdiag_debug 025 failed in run_test failed
Test control_thompson_debug 023 failed failed
Test control_thompson_debug 023 failed in run_test failed
Test control_rrtmgp_debug 026 failed failed
Test control_rrtmgp_debug 026 failed in run_test failed
Test fv3_gsd_diag_debug 022 failed failed
Test fv3_gsd_diag_debug 022 failed in run_test failed
Test fv3_rrfs_v1alpha_debug 019 failed failed
Test control_thompson_no_aero_debug 024 failed failed
Test fv3_rrfs_v1alpha_debug 019 failed in run_test failed
Test control_thompson_no_aero_debug 024 failed in run_test failed
Please make changes and add the following label back:
cheyenne-gnu-BL

BrianCurtis-NOAA · 2021-08-26T14:30:24Z

Automated RT Failure Notification
Machine: gaea
Compiler: intel
Job: BL
Repo location: /lustre/f2/pdata/ncep/emc.nemspara/autort/pr/720092158/20210826133005/ufs-weather-model
Please manually delete: /lustre/f2/scratch/emc.nemspara/FV3_RT/rt_26714
Test control_thompson_no_aero 030 failed failed
Test control_thompson_no_aero 030 failed in run_test failed
Test control_csawmgt 025 failed failed
Test control_csawmgt 025 failed in run_test failed
Test regional_quilt_hafs 035 failed failed
Test regional_quilt 034 failed failed
Test regional_quilt 034 failed in run_test failed
Test regional_quilt_hafs 035 failed in run_test failed
Test control_c384 016 failed failed
Test control_c384 016 failed in run_test failed
Please make changes and add the following label back:
gaea-intel-BL

climbfuji · 2021-08-26T14:38:17Z

Machine: cheyenne
Compiler: gnu
Job: BL
Repo location: /glade/scratch/dtcufsrt/autort/tests/auto/pr/720092158/20210826073009/ufs-weather-model
Please manually delete: /glade/scratch/dtcufsrt/FV3_RT/rt_10056
Baseline creation and move successful
Repo location: /glade/scratch/dtcufsrt/autort/tests/auto/pr/720092158/20210826075727/ufs-weather-model
Please manually delete: /glade/scratch/dtcufsrt/FV3_RT/rt_45380
Test fv3_esg_HAFS_v0_hwrf_thompson 015 failed failed
Test fv3_esg_HAFS_v0_hwrf_thompson 015 failed in run_test failed
Test control_ugwpv1 009 failed failed
Test control_ugwpv1 009 failed in run_test failed
Test control_thompson 007 failed failed
Test control_thompson 007 failed in run_test failed
Test control 001 failed failed
Test control 001 failed in run_test failed
Test control_thompson_no_aero 008 failed failed
Test control_thompson_no_aero 008 failed in run_test failed
Test fv3_gsd 011 failed failed
Test fv3_gsd 011 failed in run_test failed
Test control_stochy 004 failed failed
Test control_rrtmgp 006 failed failed
Test control_stochy 004 failed in run_test failed
Test control_rrtmgp 006 failed in run_test failed
Test control_flake 005 failed failed
Test control_flake 005 failed in run_test failed
Test control_ras 010 failed failed
Test control_ras 010 failed in run_test failed
Test control_ras_debug 028 failed failed
Test control_ras_debug 028 failed in run_test failed
Test control_noahmp_debug 029 failed failed
Test control_noahmp_debug 029 failed in run_test failed
Test control_debug 016 failed failed
Test control_debug 016 failed in run_test failed
Test control_diag_debug 017 failed failed
Test control_diag_debug 017 failed in run_test failed
Test fv3_gsd_debug 021 failed failed
Test fv3_gsd_debug 021 failed in run_test failed
Test control_c48 003 failed failed
Test control_c48 003 failed in run_test failed
Test fv3_rrfs_v1beta_debug 020 failed failed
Test fv3_rrfs_v1beta_debug 020 failed in run_test failed
Test control_ugwpv1_debug 027 failed failed
Test control_ugwpv1_debug 027 failed in run_test failed
Test control_thompson_extdiag_debug 025 failed failed
Test control_thompson_extdiag_debug 025 failed in run_test failed
Test control_thompson_debug 023 failed failed
Test control_thompson_debug 023 failed in run_test failed
Test control_rrtmgp_debug 026 failed failed
Test control_rrtmgp_debug 026 failed in run_test failed
Test fv3_gsd_diag_debug 022 failed failed
Test fv3_gsd_diag_debug 022 failed in run_test failed
Test fv3_rrfs_v1alpha_debug 019 failed failed
Test control_thompson_no_aero_debug 024 failed failed
Test fv3_rrfs_v1alpha_debug 019 failed in run_test failed
Test control_thompson_no_aero_debug 024 failed in run_test failed
Please make changes and add the following label back:
cheyenne-gnu-BL

Weird. It says baseline creation was successful, and it looks like it was. For the verification step, I see the following error for the failing tests:

0: in fcst run phase 2, na=           0
144: line          570 NetCDF: HDF error
144:MPT ERROR: Rank 144(g:144) is aborting with error code 1.
144:    Process ID: 26133, Host: r10i6n26, Program: /glade/scratch/dtcufsrt/FV3_RT/rt_45380/control_noahmp_debug/fv3.exe
144:    MPT Version: HPE MPT 2.22  03/31/20 15:59:10
144:
144:MPT: --------stack traceback-------

Not sure if this was a temporary issue, a disk space problem or what. Will try to run the verification step manually.

junwang-noaa · 2021-08-26T14:40:38Z

On gaea, all the 4 failed tests were timed out. Will manually resubmit the 4 tests for baseline creation.

…

On Thu, Aug 26, 2021 at 10:30 AM Brian Curtis ***@***.***> wrote: *Automated RT Failure Notification* Machine: gaea Compiler: intel Job: BL Repo location: /lustre/f2/pdata/ncep/emc.nemspara/autort/pr/720092158/20210826133005/ufs-weather-model Please manually delete: /lustre/f2/scratch/emc.nemspara/FV3_RT/rt_26714 Test control_thompson_no_aero 030 failed failed Test control_thompson_no_aero 030 failed in run_test failed Test control_csawmgt 025 failed failed Test control_csawmgt 025 failed in run_test failed Test regional_quilt_hafs 035 failed failed Test regional_quilt 034 failed failed Test regional_quilt 034 failed in run_test failed Test regional_quilt_hafs 035 failed in run_test failed Test control_c384 016 failed failed Test control_c384 016 failed in run_test failed Please make changes and add the following label back: gaea-intel-BL — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#772 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI7D6TOCWOD2KD4FJCKQPMLT6ZFYXANCNFSM5C2HHSNQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

climbfuji · 2021-08-26T14:42:41Z

On gaea, all the 4 failed tests were timed out. Will manually resubmit the 4 tests for baseline creation.
…

@bensonr Do we expect an increase in runtime from the dycore update?

on-behalf-of @ufs-community

bensonr · 2021-08-26T14:47:52Z

On gaea, all the 4 failed tests were timed out. Will manually resubmit the 4 tests for baseline creation. … <#m_8415720038615092642_> @bensonr <https://github.com/bensonr> Do we expect an increase in runtime from the dycore update?

No I don't. I was able to run a full set of tests without fail on gaea and both intel and gnu on hera last night.

BrianCurtis-NOAA · 2021-08-26T14:48:18Z

Machine: cheyenne
Compiler: intel
Job: BL
Repo location: /glade/scratch/dtcufsrt/autort/tests/auto/pr/720092158/20210826074509/ufs-weather-model
Please manually delete: /glade/scratch/dtcufsrt/FV3_RT/rt_67736
Test fv3_gsd_diag_debug 065 failed in run_test failed
Test datm_control_cfsr 075 failed in run_test failed
Test compile 013 failed failed
Test datm_control_gefs 076 failed in run_test failed
Test datm_control_iau_gefs 077 failed in run_test failed
Test datm_bulk_cfsr 078 failed in run_test failed
Test datm_bulk_gefs 079 failed in run_test failed
Test datm_mx025_cfsr 080 failed in run_test failed
Test datm_mx025_gefs 081 failed in run_test failed
Test cpld_control_c192 003 failed failed
Test cpld_control_c192 003 failed in run_test failed
Test regional_quilt 034 failed failed
Test regional_quilt 034 failed in run_test failed
Test cpld_bmark_v16 005 failed failed
Test cpld_bmark_v16 005 failed in run_test failed
Test datm_cdeps_bulk_cfsr 085 failed in run_test failed
Test datm_cdeps_bulk_gefs 086 failed in run_test failed
Test datm_cdeps_mx025_cfsr 087 failed in run_test failed
Test datm_cdeps_mx025_gefs 088 failed in run_test failed
Test datm_cdeps_debug_cfsr 089 failed in run_test failed
Test fv3_esg_HAFS_v0_hwrf_thompson 044 failed failed
Test fv3_esg_HAFS_v0_hwrf_thompson 044 failed in run_test failed
Test cpld_control_wave 008 failed failed
Test cpld_control_wave 008 failed in run_test failed
Test cpld_bmark_wave_v16 007 failed failed
Test cpld_bmark_wave_v16 007 failed in run_test failed
Test cpld_bmark_v16_nsst 006 failed failed
Test cpld_bmark_v16_nsst 006 failed in run_test failed
Test compile 019 failed failed
Test datm_debug_cfsr 082 failed failed
Test datm_debug_cfsr 082 failed in run_test failed
Test datm_cdeps_control_cfsr 083 failed failed
Test datm_cdeps_control_cfsr 083 failed in run_test failed
Test control_rrtmgp 023 failed failed
Test control_rrtmgp 023 failed in run_test failed
Test regional_quilt_hafs 035 failed failed
Test regional_quilt_hafs 035 failed in run_test failed
Test control_csawmg 024 failed failed
Test control_csawmg 024 failed in run_test failed
Test compile 014 failed failed
Test datm_cdeps_control_gefs 084 failed failed
Test datm_cdeps_control_gefs 084 failed in run_test failed
Test regional_quilt_RRTMGP 037 failed failed
Test regional_quilt_RRTMGP 037 failed in run_test failed
Test regional_quilt_netcdf_parallel 036 failed failed
Test regional_quilt_netcdf_parallel 036 failed in run_test failed
Test control_csawmgt 025 failed failed
Test control_csawmgt 025 failed in run_test failed
Test control_flake 026 failed failed
Test control_ugwpv1 027 failed failed
Test control_flake 026 failed in run_test failed
Test control_ugwpv1 027 failed in run_test failed
Test control_thompson 029 failed failed
Test control_ras 028 failed failed
Test control_thompson 029 failed in run_test failed
Test control_ras 028 failed in run_test failed
Test control_thompson_no_aero 030 failed failed
Test control_thompson_no_aero 030 failed in run_test failed
Test control_noahmp 031 failed failed
Test control_noahmp 031 failed in run_test failed
Test regional_control 032 failed failed
Test regional_control 032 failed in run_test failed
Test fv3_gsd 038 failed failed
Test fv3_gsd 038 failed in run_test failed
Test fv3_rrfs_v1alpha 039 failed failed
Test fv3_rrfs_v1alpha 039 failed in run_test failed
Test fv3_rap 040 failed failed
Test fv3_rap 040 failed in run_test failed
Test fv3_rrfs_v1beta 042 failed failed
Test fv3_rrfs_v1beta 042 failed in run_test failed
Test fv3_hrrr 041 failed failed
Test fv3_hrrr 041 failed in run_test failed
Test cpld_control_c384 004 failed failed
Test cpld_control_c384 004 failed in run_test failed
Test control_c192 014 failed failed
Test control_c192 014 failed in run_test failed
Test control_c384 015 failed failed
Test control_c384 015 failed in run_test failed
Test control_c384gdas 016 failed failed
Test control_c384gdas 016 failed in run_test failed
Please make changes and add the following label back:
cheyenne-intel-BL

…core

on-behalf-of @ufs-community

BrianCurtis-NOAA · 2021-08-26T16:47:54Z

Automated RT Failure Notification
Machine: orion
Compiler: intel
Job: BL
Repo location: /work/noaa/nems/emc.nemspara/autort/pr/720092158/20210826110011/ufs-weather-model
Please manually delete: /work/noaa/stmp/bcurtis/stmp/bcurtis/FV3_RT/rt_53807
Please make changes and add the following label back:
orion-intel-BL

BrianCurtis-NOAA · 2021-08-26T18:06:22Z

Automated RT Failure Notification
Machine: gaea
Compiler: intel
Job: RT
Repo location: /lustre/f2/pdata/ncep/emc.nemspara/autort/pr/720092158/20210826164508/ufs-weather-model
Please manually delete: /lustre/f2/scratch/emc.nemspara/FV3_RT/rt_9846
Test control 017 failed failed
Test control 017 failed in run_test failed
Test control_merra2 034 failed failed
Test control_merra2 034 failed in run_test failed
Test regional_restart 044 failed failed
Test regional_restart 044 failed in run_test failed
Please make changes and add the following label back:
gaea-intel-RT

MinsukJi-NOAA · 2021-08-26T18:15:09Z

All tests that failed on gaea were timed out. For example, control_merra2 ran for 15 hours instead of 24 hours. I will manually re-run those 3 tests.

junwang-noaa · 2021-08-26T18:16:51Z

Thanks, Minsuk.

…

On Thu, Aug 26, 2021 at 2:15 PM Minsuk Ji ***@***.***> wrote: All tests that failed on gaea were timed out. For example, control_merra2 has ran for 15 hours instead of 24 hours. I will manually re-run those 3 tests. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#772 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI7D6TNKPQW7PQGIVQAI3YTT62ADRANCNFSM5C2HHSNQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

junwang-noaa · 2021-08-26T20:03:49Z

@BrianCurtis-NOAA The orion baseline was created successfully, but the baseline was not copied to the /work/noaa/nems/emc.nemspara/RT/NEMSfv3gfs/develop-20210825/INTEL, would you please manually copy them, then change permission so that all the developers can see it? Thanks

BrianCurtis-NOAA · 2021-08-26T20:08:26Z

@BrianCurtis-NOAA The orion baseline was created successfully, but the baseline was not copied to the /work/noaa/nems/emc.nemspara/RT/NEMSfv3gfs/develop-20210825/INTEL, would you please manually copy them, then change permission so that all the developers can see it? Thanks

Sure thing. I've started the rsync. Once completed I'll add the orion intel RT label.

junwang-noaa · 2021-08-26T20:20:52Z

@bensonr the wcoss dell/cray log files are on hera at:

/scratch1/NCEPDEV/stmp2/Jun.Wang/wcosslog
RegressionTests_wcoss_cray.log  RegressionTests_wcoss_dell_p3.log

Please commit them to your branch

bensonr · 2021-08-26T21:03:45Z

@junwang-noaa - wcoss regression logs are in place

on-behalf-of @ufs-community

BrianCurtis-NOAA · 2021-08-27T01:25:46Z

Automated RT Failure Notification
Machine: jet
Compiler: intel
Job: BL
Repo location: /lfs4/HFIP/h-nems/emc.nemspara/autort/pr/720092158/20210826133015/ufs-weather-model
Please manually delete: /lfs4/HFIP/h-nems/emc.nemspara/RT_RUNDIRS/emc.nemspara/FV3_RT/rt_17906
Baseline creation and move successful
Repo location: /lfs4/HFIP/h-nems/emc.nemspara/autort/pr/720092158/20210826174959/ufs-weather-model
Please manually delete: /lfs4/HFIP/h-nems/emc.nemspara/RT_RUNDIRS/emc.nemspara/FV3_RT/rt_51597
Test control_2threads 018 failed failed
Test control_2threads 018 failed in run_test failed
Please make changes and add the following label back:
jet-intel-BL

climbfuji

Approved contingent on jet.intel tests passing (and the usual submodule/.gitmodules procedure).

BrianCurtis-NOAA · 2021-08-27T13:10:13Z

With current Jet issues, you may want to skip it for this PR.

junwang-noaa · 2021-08-27T13:17:27Z

I have jet log file ready, but now jet is not accessible. Will attach to the PR when jet is back.

…core

bensonr added 2 commits August 25, 2021 20:22

updated gitmodules and FV3 to point to bensonr/fv3atm::dycore

58f4ccf

* updated ICs to use newly created 20210825 snapshot

43f69d4

* bypass control_csawmg test for gaea (numerical instability)

bensonr changed the title ~~Dycore~~ brings in FV3 "the dycore" updated to be consistent with the latest public release Aug 26, 2021

junwang-noaa added Baseline Updates Current baselines will be updated. input data change Input data change labels Aug 26, 2021

junwang-noaa requested review from climbfuji and junwang-noaa August 26, 2021 12:30

updated BL_DATE in anticipation of baseline creation

f4ac245

junwang-noaa added gaea-intel-BL labels Aug 26, 2021

BrianCurtis-NOAA removed gaea-intel-BL labels Aug 26, 2021

RT JOBS PASSED: hera.gnu. Log file uploaded.

57e5367

on-behalf-of @ufs-community

BrianCurtis-NOAA removed the orion-intel-BL label Aug 26, 2021

junwang-noaa added the gaea-intel-RT label Aug 26, 2021

bensonr added 2 commits August 26, 2021 12:34

updates requested for container CI testing

d94d1b7

Merge branch 'dycore' of github.com:bensonr/ufs-weather-model into dy…

a868fa0

…core

MinsukJi-NOAA added the run-ci label Aug 26, 2021

github-actions bot removed the run-ci label Aug 26, 2021

BrianCurtis-NOAA removed the gaea-intel-RT label Aug 26, 2021

RT JOBS PASSED: cheyenne.intel. Log file uploaded.

a788cbb

on-behalf-of @ufs-community

Add gaea RT log: all tests passed

42dc050

BrianCurtis-NOAA added orion-intel-RT and removed orion-intel-RT labels Aug 26, 2021

Add wcoss cray and dell_p3 RT logs: all tests passed

130c749

RT JOBS PASSED: orion.intel. Log file uploaded.

38e6e62

on-behalf-of @ufs-community

climbfuji approved these changes Aug 27, 2021

View reviewed changes

junwang-noaa approved these changes Aug 27, 2021

View reviewed changes

bensonr added 2 commits August 27, 2021 11:13

revert .gitmodules and update FV3 hash and remote

3ebb690

Merge branch 'dycore' of github.com:bensonr/ufs-weather-model into dy…

c58574e

…core

junwang-noaa merged commit ef2d0ed into ufs-community:develop Aug 27, 2021

bensonr deleted the dycore branch October 14, 2021 12:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

brings in FV3 "the dycore" updated to be consistent with the latest public release #772

brings in FV3 "the dycore" updated to be consistent with the latest public release #772

bensonr commented Aug 26, 2021 •

edited by junwang-noaa

Loading

junwang-noaa commented Aug 26, 2021 •

edited

Loading

bensonr commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

climbfuji commented Aug 26, 2021

junwang-noaa commented Aug 26, 2021 via email

climbfuji commented Aug 26, 2021

bensonr commented Aug 26, 2021 via email

BrianCurtis-NOAA commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

MinsukJi-NOAA commented Aug 26, 2021 •

edited

Loading

junwang-noaa commented Aug 26, 2021 via email

junwang-noaa commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

junwang-noaa commented Aug 26, 2021

bensonr commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 27, 2021

climbfuji left a comment

BrianCurtis-NOAA commented Aug 27, 2021

junwang-noaa commented Aug 27, 2021

brings in FV3 "the dycore" updated to be consistent with the latest public release #772

brings in FV3 "the dycore" updated to be consistent with the latest public release #772

Conversation

bensonr commented Aug 26, 2021 • edited by junwang-noaa Loading

PR Checklist

Instructions: All subsequent sections of text should be filled in as appropriate.

Description

Issue(s) addressed

Testing

Dependencies

junwang-noaa commented Aug 26, 2021 • edited Loading

bensonr commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

climbfuji commented Aug 26, 2021

junwang-noaa commented Aug 26, 2021 via email

climbfuji commented Aug 26, 2021

bensonr commented Aug 26, 2021 via email

BrianCurtis-NOAA commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

MinsukJi-NOAA commented Aug 26, 2021 • edited Loading

junwang-noaa commented Aug 26, 2021 via email

junwang-noaa commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 26, 2021

junwang-noaa commented Aug 26, 2021

bensonr commented Aug 26, 2021

BrianCurtis-NOAA commented Aug 27, 2021

climbfuji left a comment

Choose a reason for hiding this comment

BrianCurtis-NOAA commented Aug 27, 2021

junwang-noaa commented Aug 27, 2021

bensonr commented Aug 26, 2021 •

edited by junwang-noaa

Loading

junwang-noaa commented Aug 26, 2021 •

edited

Loading

MinsukJi-NOAA commented Aug 26, 2021 •

edited

Loading