Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update sorc/gdas.cd hash #3219

Closed
2 tasks done
RussTreadon-NOAA opened this issue Jan 10, 2025 · 7 comments · Fixed by #3220
Closed
2 tasks done

Update sorc/gdas.cd hash #3219

RussTreadon-NOAA opened this issue Jan 10, 2025 · 7 comments · Fixed by #3220
Assignees
@RussTreadon-NOAA
Labels
feature New feature or request

Comments

@RussTreadon-NOAA
Copy link
Contributor

RussTreadon-NOAA commented Jan 10, 2025 •
edited
Loading

What new functionality do you need?

g-w develop currently points at gdas.cd @ d6097af. This GDASApp hash is from mid-December 2025.

GDASApp develop has been updated multiple times since then. For example,

  • several JEDI hashes have been updated
  • the WCOSS2 build has been updated to spack-stack/1.6.0

Developers want to exercise these new capabilities from g-w.

What are the requirements for the new functionality?

GDASApp should run within g-w on WCOSS2 and select NOAA RHPCS machines (Hera, Hercules, Orion)

Acceptance Criteria

  • GDASApp builds on WCOSS2, Hera, Hercules, and Orion
  • g-w CI successfully runs to completion on WCOSS2, Hera, Hercules, and Orion

Suggest a solution (optional)

No response
@RussTreadon-NOAA RussTreadon-NOAA added feature New feature or request triage Issues that are triage labels Jan 10, 2025
@RussTreadon-NOAA RussTreadon-NOAA self-assigned this Jan 10, 2025
@RussTreadon-NOAA RussTreadon-NOAA mentioned this issue Jan 10, 2025
Closed
@RussTreadon-NOAA
Copy link
Contributor Author

Work for this issue will be done in RussTreadon-NOAA:feature/update_gdas

RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Jan 10, 2025
@RussTreadon-NOAA
update sorc/gdas.cd hash (NOAA-EMC#3219)
228eb52
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Jan 10, 2025
@RussTreadon-NOAA
updates required to run GDASApp using spack-stack/1.6.0 on WCOSS2 (NO…
d7e46c0 
…AA-EMC#3219)
@RussTreadon-NOAA RussTreadon-NOAA mentioned this issue Jan 10, 2025
Merged
8 tasks
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Jan 10, 2025
@RussTreadon-NOAA
fix shellnorm violations (NOAA-EMC#3219)
4e73a31
@RussTreadon-NOAA
Copy link
Contributor Author

Successfully build feature/update_gdas at 4e73a31 on WCOSS2 (Cactus), Hera, Hercules, and Orion. Set up and start g-w CI on each machine from user account.

@RussTreadon-NOAA
Copy link
Contributor Author

g-w CI

Install feature/update_gdas at 4e73a31 on WCOSS2 (Cactus), Hera, Hercules, and Orion. Run g-w CI with the following results on each machine

WCOSS2 (Cactus)

/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_ATM_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Jan 10 2025 21:04:16    Jan 10 2025 22:20:24
202103231800        Done    Jan 10 2025 21:04:16    Jan 10 2025 22:30:30
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_3DVarAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Jan 10 2025 21:04:18    Jan 10 2025 21:25:29
202103250000        Done    Jan 10 2025 21:04:18    Jan 10 2025 22:25:32
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48mx500_hybAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Jan 10 2025 21:04:19    Jan 10 2025 21:25:32
202103250000        Done    Jan 10 2025 21:04:19    Jan 10 2025 22:30:35
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SWA_gefs_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Jan 10 2025 21:04:24    Jan 10 2025 23:55:37
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C48_S2SW_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Jan 10 2025 21:04:26    Jan 10 2025 22:50:32
202103231800        Done    Jan 10 2025 21:04:26    Jan 10 2025 23:00:22
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_atm3DVar_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Jan 10 2025 21:04:28    Jan 10 2025 21:25:40
202112210000        Done    Jan 10 2025 21:04:28    Jan 10 2025 23:35:33
202112210600        Done    Jan 10 2025 21:04:28    Jan 10 2025 23:20:37
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmaerosnowDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Jan 10 2025 21:04:30    Jan 10 2025 21:30:53
202112201800        Done    Jan 10 2025 21:04:30    Jan 10 2025 23:35:36
202112210000        Done    Jan 10 2025 21:04:30    Jan 10 2025 23:25:34
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Jan 10 2025 21:04:32    Jan 10 2025 21:25:47
202112210000        Done    Jan 10 2025 21:04:32    Jan 10 2025 23:35:39
202112210600        Done    Jan 10 2025 21:04:32    Jan 10 2025 23:20:43
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96C48_ufs_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202402231800        Done    Jan 10 2025 21:04:33    Jan 10 2025 21:25:50
202402240000        Done    Jan 10 2025 21:04:33    Jan 11 2025 00:05:39
202402240600        Done    Jan 10 2025 21:04:33    Jan 11 2025 00:00:38
 
/lfs/h2/emc/ptmp/russ.treadon/EXPDIR/C96_S2SWA_gefs_replay_ics_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202011010000        Done    Jan 10 2025 21:04:36    Jan 10 2025 22:21:02

Hera

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C48_ATM_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 20:52:10    Jan 10 2025 22:10:17
202103231800        Done    Jan 10 2025 20:52:10    Jan 10 2025 22:20:16

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C48_S2SW_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 20:52:13    Jan 11 2025 00:20:40
202103231800        Done    Jan 10 2025 20:52:13    Jan 11 2025 00:30:26

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C48_S2SWA_gefs_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 20:52:19    Jan 11 2025 00:50:22

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C48mx500_3DVarAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 20:52:24    Jan 10 2025 21:30:29
202103250000        Done    Jan 10 2025 20:52:24    Jan 10 2025 22:45:28

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C48mx500_hybAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 20:52:27    Jan 10 2025 21:30:31
202103250000        Done    Jan 10 2025 20:52:27    Jan 10 2025 22:40:30

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C96C48_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201800        Done    Jan 10 2025 20:52:30    Jan 10 2025 21:35:29
202112210000        Done    Jan 10 2025 20:52:30    Jan 11 2025 00:05:29
202112210600        Done    Jan 10 2025 20:52:30    Jan 10 2025 23:45:32

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C96C48_hybatmaerosnowDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201200        Done    Jan 10 2025 20:52:33    Jan 10 2025 21:40:32
202112201800        Done    Jan 10 2025 20:52:33    Jan 11 2025 00:01:37
202112210000        Done    Jan 10 2025 20:52:33    Jan 10 2025 23:50:33

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C96C48_ufs_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202402231800        Done    Jan 10 2025 20:52:36    Jan 10 2025 21:40:34
202402240000        Done    Jan 10 2025 20:52:36    Jan 11 2025 00:40:36
202402240600        Done    Jan 10 2025 20:52:36    Jan 11 2025 00:36:18

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C96_S2SWA_gefs_replay_ics_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202011010000        Done    Jan 10 2025 20:52:40    Jan 10 2025 22:55:33

/scratch1/NCEPDEV/stmp2/Russ.Treadon/EXPDIR/C96_atm3DVar_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201800        Done    Jan 10 2025 20:52:43    Jan 10 2025 21:40:38
202112210000        Done    Jan 10 2025 20:52:43    Jan 11 2025 00:15:42
202112210600        Done    Jan 10 2025 20:52:43    Jan 10 2025 23:40:33

Hercules
Batch job throughput is extremely slow on Hercules. Only two g-w CI suites have completed as of 1/11/2025 morning. 120 jobs are PENDING in the batch queue.

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48_ATM_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 21:10:03    Jan 11 2025 04:00:04
202103231800        Done    Jan 10 2025 21:10:03    Jan 11 2025 04:20:04

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48mx500_3DVarAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 21:10:04    Jan 10 2025 21:45:04
202103250000        Done    Jan 10 2025 21:10:04    Jan 11 2025 09:00:05

Unfortunately, there is one job failure on Hercules

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48mx500_hybAOWCDA_update_gdas
202103250000     gdas_marineanlletkf                     4218366                DEAD                   1         2         118.0

Orion

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C48_ATM_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 21:18:10    Jan 10 2025 22:30:06
202103231800        Done    Jan 10 2025 21:18:10    Jan 10 2025 22:50:05

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C48mx500_3DVarAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 21:18:12    Jan 10 2025 21:40:07
202103250000        Done    Jan 10 2025 21:18:12    Jan 10 2025 22:55:06

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C48mx500_hybAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 21:18:14    Jan 10 2025 21:40:09
202103250000        Done    Jan 10 2025 21:18:14    Jan 10 2025 23:00:10

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C48_S2SWA_gefs_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 21:18:23    Jan 11 2025 01:20:28

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C48_S2SW_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 21:18:26    Jan 11 2025 00:55:10
202103231800        Done    Jan 10 2025 21:18:26    Jan 11 2025 01:20:11

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C96_atm3DVar_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201800        Done    Jan 10 2025 21:18:28    Jan 10 2025 21:40:15
202112210000        Done    Jan 10 2025 21:18:28    Jan 11 2025 00:20:13
202112210600        Done    Jan 10 2025 21:18:28    Jan 11 2025 00:05:12

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C96C48_hybatmaerosnowDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201200        Done    Jan 10 2025 21:18:30    Jan 10 2025 21:40:17
202112201800        Done    Jan 10 2025 21:18:30    Jan 11 2025 00:05:14
202112210000        Done    Jan 10 2025 21:18:30    Jan 11 2025 00:10:14

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C96C48_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201800        Done    Jan 10 2025 21:18:32    Jan 10 2025 21:40:19
202112210000        Done    Jan 10 2025 21:18:32    Jan 11 2025 00:15:15
202112210600        Done    Jan 10 2025 21:18:32    Jan 11 2025 00:10:16

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C96C48_ufs_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202402231800        Done    Jan 10 2025 21:18:34    Jan 10 2025 21:40:22
202402240000        Done    Jan 10 2025 21:18:34    Jan 11 2025 01:20:19
202402240600        Done    Jan 10 2025 21:18:34    Jan 11 2025 02:05:17

/work2/noaa/stmp/rtreadon/ORION/EXPDIR/C96_S2SWA_gefs_replay_ics_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202011010000        Done    Jan 10 2025 21:18:37    Jan 10 2025 23:00:36

Summary

  1. All jobs in all g-w CI streams successfully ran to completion on WCOSS2 (Cactus), Hera, and Orion.
  2. g-w CI is still running on Hercules. There is one failure - 20210325 00Z gdas_marineanlletkf in C48mx500_hybAOWCDA. This failure appears to be due to a missing job dependency (see comment below)

@RussTreadon-NOAA
Copy link
Contributor Author

Hercules C48mx500_hybAOWCDA failure

Examination of the log file for the 20210325 00Z gdas_marineanlletkf shows that the job failed due to a missing file

 8:
 8: FATAL from PE     8: fms_io(restore_state_all): unable to find any restart files specified by ../ensdata/ens/ocean.1.nc
 8:
 8: Image              PC                Routine            Line        Source
 8: libsoca.so         000014E819CDC909  mpp_mod_mp_mpp_er          72  mpp_util_mpi.inc
 8: libsoca.so         000014E819BB0E58  fms_io_mod_mp_res        4094  fms_io.F90
 8: libsoca.so         000014E819A1BFD2  Unknown               Unknown  Unknown
 8: libsoca.so         000014E819B19944  Unknown               Unknown  Unknown
 8: libsoca.so         000014E819B1D3DB  Unknown               Unknown  Unknown
 8: gdas.x             00000000007214A3  Unknown               Unknown  Unknown
 8: gdas.x             00000000007212DF  Unknown               Unknown  Unknown
 8: gdas.x             000000000085A2F3  Unknown               Unknown  Unknown
 8: gdas.x             000000000074AD7C  Unknown               Unknown  Unknown
 8: gdas.x             000000000074A727  Unknown               Unknown  Unknown
 8: gdas.x             00000000007438F0  Unknown               Unknown  Unknown
 8: liboops.so         000014E7ECCAF152  _ZN4oops3Run7exec     Unknown  Unknown
 8: gdas.x             00000000004D9762  Unknown               Unknown  Unknown

File ensdata/ens/ocean.1.nc is created by job gdas_marinebmat. The gdas_marinebmat log file contains

gdas_marinebmat.log:2025-01-10 22:39:58,072 - INFO     - file_utils  : Copied /work2/noaa/stmp/rtreadon/HERCULES/COMROOT/C48mx500_hybAOWCDA_update_gdas/enkfgdas.20210324/18/mem001/model/ocean/history/enkfgdas.ocean.t18z.inst.f006.nc to /work/noaa/stmp/rtreadon/HERCULES/RUNDIRS/C48mx500_hybAOWCDA_update_gdas/gdas.2021032500/gdasmarineanalysis.2021032500/ensdata/ens/ocean.1.nc

The gdas_marineanlletkf job ran before the gdas_marinebmat job

hercules-login-3:/work2/noaa/stmp/rtreadon/HERCULES/COMROOT/C48mx500_hybAOWCDA_update_gdas/logs/2021032500$ ls -lt gdas_marinebmat.log gdas_marineanlletkf.log
-rw-r--r-- 1 rtreadon stmp 3358955 Jan 10 22:40 gdas_marinebmat.log
-rw-r--r-- 1 rtreadon stmp  309322 Jan 10 22:14 gdas_marineanlletkf.log

The xml file, C48mx500_hybAOWCDA_update_gdas.xml, has the following dependencies for gdas_marineanlletkf

        <dependency>
                <and>
                        <metataskdep metatask="enkfgdas_fcst" cycle_offset="-06:00:00"/>
                        <taskdep task="gdas_prepoceanobs"/>
                </and>
        </dependency>

gdas_marinebmat is not listed as a job dependency.

@guillaumevernieres and @AndrewEichmann-NOAA : Does gdas_marineanlletkf require that gdas_marinebmat successfully complete before gdas_marineanlletkf can be submitted?

If yes, we can add that this dependency to g-w PR #3220.

@RussTreadon-NOAA
Copy link
Contributor Author

Interesting observation
Although the Hercules 20210325 00Z gdas_marineanlletkf failed, all the other jobs in the C48mx500_hybAOWCDA suite successfully ran to completion.

@RussTreadon-NOAA
Copy link
Contributor Author

g-w CI on Hercules

g-w CI completed on Hercules with the following results

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48_ATM_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 21:10:03    Jan 11 2025 04:00:04
202103231800        Done    Jan 10 2025 21:10:03    Jan 11 2025 04:20:04

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48_S2SW_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 21:10:06    Jan 11 2025 22:55:06
202103231800        Done    Jan 10 2025 21:10:06    Jan 11 2025 23:15:06

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48_S2SWA_gefs_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103231200        Done    Jan 10 2025 21:10:20    Jan 12 2025 01:50:14

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48mx500_3DVarAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 21:10:04    Jan 10 2025 21:45:04
202103250000        Done    Jan 10 2025 21:10:04    Jan 11 2025 09:00:05

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48mx500_hybAOWCDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 21:10:05    Jan 10 2025 22:40:06
202103250000      Active    Jan 10 2025 21:10:05             -

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C96C48_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201800        Done    Jan 10 2025 21:10:15    Jan 10 2025 23:35:08
202112210000        Done    Jan 10 2025 21:10:15    Jan 11 2025 15:05:09
202112210600        Done    Jan 10 2025 21:10:15    Jan 11 2025 21:45:10

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C96C48_hybatmaerosnowDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201200        Done    Jan 10 2025 21:10:09    Jan 10 2025 23:55:08
202112201800        Done    Jan 10 2025 21:10:09    Jan 11 2025 15:00:09
202112210000        Done    Jan 10 2025 21:10:09    Jan 11 2025 22:00:10

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C96C48_ufs_hybatmDA_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202402231800        Done    Jan 10 2025 21:10:16    Jan 10 2025 23:50:12
202402240000        Done    Jan 10 2025 21:10:16    Jan 11 2025 16:55:13
202402240600        Done    Jan 10 2025 21:10:16    Jan 11 2025 18:25:09

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C96_S2SWA_gefs_replay_ics_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202011010000        Done    Jan 10 2025 21:10:21    Jan 11 2025 22:35:19

/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C96_atm3DVar_update_gdas
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202112201800        Done    Jan 10 2025 21:10:07    Jan 10 2025 22:45:08
202112210000        Done    Jan 10 2025 21:10:07    Jan 11 2025 14:10:07
202112210600        Done    Jan 10 2025 21:10:07    Jan 11 2025 17:25:08

All jobs in all streams except C48mx500_hybAOWCDA successfully ran to completion.

As explained above, job gdas_marineanlletkf failed in C48mx500_hybAOWCDA due to the job running before required input was available. gdas_marinebmat successfully completed after gdas_marineanlletkf failed.

Rewind and reboot gdas_marineanlletkf. This attempt failed because the expected input from gdas_marinebmat resides in a run directory. Rewind and reboot gdas_marinebmat. Job successfully completed. Rewind and reboot gdas_marineanlletkf. This time the job successfully completed.

hercules-login-3:/work2/noaa/stmp/rtreadon/HERCULES/EXPDIR/C48mx500_hybAOWCDA_update_gdas$ rocotostat -d C48mx500_hybAOWCDA_update_gdas.db -w C48mx500_hybAOWCDA_update_gdas.xml -c all -s
   CYCLE         STATE           ACTIVATED              DEACTIVATED
202103241800        Done    Jan 10 2025 21:10:05    Jan 10 2025 22:40:06
202103250000        Done    Jan 10 2025 21:10:05    Jan 13 2025 00:40:05

@RussTreadon-NOAA
Copy link
Contributor Author

RussTreadon-NOAA commented Jan 13, 2025 •
edited
Loading

PR #3220 is ready for review.

RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Jan 13, 2025
@RussTreadon-NOAA
Add TODO comments to WCOSS2.env (NOAA-EMC#3219)
ce98ffc
@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the triage Issues that are triage label Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RussTreadon-NOAA RussTreadon-NOAA

Labels
feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update the gdas.cd hash and enable GDASApp to run on WCOSS2 RussTreadon-NOAA/global-workflow
2 participants
@RussTreadon-NOAA @WalterKolczynski-NOAA