Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stage atmospheric backgrounds and UFS cubed-sphere history files #2792

Merged
merged 122 commits into from
Aug 8, 2024
Merged

Stage atmospheric backgrounds and UFS cubed-sphere history files #2792

merged 122 commits into from
Aug 8, 2024

Conversation

DavidNew-NOAA
Copy link
Contributor

@DavidNew-NOAA DavidNew-NOAA commented Jul 25, 2024 •
edited
Loading

Description

This PR changes the G-W so that backgrounds for the atmospheric analyses are staged as UFS cubed-sphere history files rather than FMS restarts. The motivation being that FMS restarts include many data we don't need for the analyses and thus take up too much disk space/bandwidth.

This is a companion PR to GDASApp #1236 and JCB-GDAS #19.

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? YES
  • Does this change require a documentation update? NO

How has this been tested?

  • Build on Hera
  • Cycling experiment on Hera
  • GDASApp jjob tests

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

danholdaway and others added 30 commits May 29, 2024 14:35
@danholdaway
Use rename_atm of jcb parm erpos
b5eb345
@DavidNew-NOAA
Initial commit
3a8f527
@DavidNew-NOAA
Update atm_berror_bump_fix.yaml.j2
6904982
@DavidNew-NOAA
Update atm_berror_gsibec_fix.yaml.j2
0b37fe5
@DavidNew-NOAA
Update atm_berror_identity_fix.yaml.j2
d770b63
@DavidNew-NOAA
Update atm_bkg_fix.yaml.j2
0187bba
@DavidNew-NOAA
Rename some things
4dcd47a
@DavidNew-NOAA
Merge branch 'feature/stage_from_yaml' of https://github.com/DavidNew…
e61963d 
…-NOAA/global-workflow into feature/stage_from_yaml
@DavidNew-NOAA
Rename some things
f91297d
@DavidNew-NOAA
Add changes to analysis python scripts
983d720
@danholdaway
update hashes and config
9436f4b
@danholdaway
Merge remote-tracking branch 'upsteam/develop' into feature/rename_atm
97a63a7 
* upsteam/develop:
  Archiving cleanup (NOAA-EMC#2621)
  Switch to Rocky 9 built external packages on Hercules (NOAA-EMC#2608)
  Add the capability to use slurm reservation nodes (NOAA-EMC#2627)
  Update forecast job to use COMIN/COMOUT (NOAA-EMC#2622)
  Update to add 1-deg global wave grid (NOAA-EMC#2619)
  Add C384mx025_3DVarAOWCDA yamls (NOAA-EMC#2625)
  Script to keep Jenkins Agent persistent from cron (NOAA-EMC#2634)
@DavidNew-NOAA
Merge branch 'develop' into feature/stage_from_yaml
6e44a5f
@DavidNew-NOAA
Merge branch 'develop' into feature/stage_from_yaml
2490049
@DavidNew-NOAA
Minor fix
3022393
@DavidNew-NOAA
Minor update
cc1a3b6
@DavidNew-NOAA
Minor update
2097116
@DavidNew-NOAA
Coding norms
5327711
@danholdaway
Remove jcb from GW submodules
2702ef0
@danholdaway
Remove JCB
b6456be
@danholdaway
adjust link to point to new jcb location in gdasapp
2a5f301
@danholdaway
Update GDASapp Hash
9f425a6
@DavidNew-NOAA
Move some stuff around
4d36caf
@danholdaway
Merge remote-tracking branch 'upsteam/develop' into feature/rename_atm
55358e5 
* upsteam/develop:
  Update ufs-weather-model  (NOAA-EMC#2646)
  Update wmo parm files to fix WMO header (NOAA-EMC#2652)
  Add IAU to snow DA (and its test) (NOAA-EMC#2610)
@danholdaway
gdas hash
9e357f9
@DavidNew-NOAA
Add time loop for some staging
ba2d12d
@danholdaway
bump gdas
d0d4429
@danholdaway
bump gdas
ee1aa30
@DavidNew-NOAA
Merge branch 'NOAA-EMC:develop' into feature/stage_from_yaml
2138df1
@danholdaway
hashes
09ec021
@WalterKolczynski-NOAA
Copy link
Contributor

Are these ocean CI tests that are failing on Hera?

Well, it isn't just an ocean test, but yes. There is a field missing from the background file:

unable to find mandatory variable aice_h in restart file ./ice.bkgerr_stddev.nc

@DavidNew-NOAA
Copy link
Contributor Author

@WalterKolczynski-NOAA OK, so I'm assuming this isn't a problem with this PR, since it doesn't have anything to do with sea ice?

@WalterKolczynski-NOAA
Copy link
Contributor

I'll try running it again. Guillaume's PR just passed, so it should be working if you didn't change anything.

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Aug 7, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Aug 7, 2024
@emcbot
Copy link

emcbot commented Aug 7, 2024

Build FAILED on Hera with error logs:

/scratch1/NCEPDEV/global/CI/2792/gfs/sorc/logs/build_ufs.log

Follow link here to view the contents of the above file(s): (link)

@emcbot emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Aug 7, 2024
@WalterKolczynski-NOAA
Copy link
Contributor

Now it's failing due to lack of space, I think.

@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed label Aug 7, 2024
@WalterKolczynski-NOAA
Copy link
Contributor

Now it's failing due to lack of space, I think.

Actually, a sysadmin killed all the CI running on Hera because we had three PRs building at once and it hammered the node. Going to relaunch a bit at a time.

@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Aug 7, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Aug 7, 2024
@emcbot
Copy link

emcbot commented Aug 8, 2024

CI Passed Hera at
Built and ran in directory /scratch1/NCEPDEV/global/CI/2792


Experiment C48_ATM_4784d241 Completed 1 Cycles: *SUCCESS* at Wed Aug  7 23:17:06 UTC 2024
Experiment C48mx500_3DVarAOWCDA_4784d241 Completed 2 Cycles: *SUCCESS* at Wed Aug  7 23:42:56 UTC 2024
Experiment C96_atm3DVar_4784d241 Completed 3 Cycles: *SUCCESS* at Thu Aug  8 00:48:40 UTC 2024
Experiment C48_S2SWA_gefs_4784d241 Completed 1 Cycles: *SUCCESS* at Thu Aug  8 00:52:02 UTC 2024
Experiment C96C48_hybatmDA_4784d241 Completed 3 Cycles: *SUCCESS* at Thu Aug  8 01:06:46 UTC 2024
Experiment C48_S2SW_4784d241 Completed 1 Cycles: *SUCCESS* at Thu Aug  8 01:11:56 UTC 2024
Experiment C96_atmaerosnowDA_4784d241 Completed 3 Cycles: *SUCCESS* at Thu Aug  8 02:05:42 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit e2c0f06 into NOAA-EMC:develop Aug 8, 2024
10 of 11 checks passed
@DavidNew-NOAA
Copy link
Contributor Author

Thank you @WalterKolczynski-NOAA

@EricSinsky-NOAA EricSinsky-NOAA mentioned this pull request Aug 8, 2024
Merged
7 tasks
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Aug 9, 2024
@DavidHuber-NOAA
Merge remote-tracking branch 'origin/develop' into fix/c768_hera
638a588 
* origin/develop:
  Stage atmospheric backgrounds and UFS cubed-sphere history files (NOAA-EMC#2792)
  Check that a PR driver is still running before trying to kill it (NOAA-EMC#2799)
  Feature/get arch adds an empty archive job to GEFS system (NOAA-EMC#2772)
  Marine DA updates (NOAA-EMC#2802)
  Revert MSU FIX_DIRs back to glopara (NOAA-EMC#2811)
  Bugfix for updating label states in Jenkins (NOAA-EMC#2808)
  Clean-up temporary rundirs - take 2. (NOAA-EMC#2753)
  Change land surface for HR4 (NOAA-EMC#2787)
  Run METplus serially and correct the name of prod tasks (NOAA-EMC#2804)
  Update Java Agent launching script for Jenkins connections (NOAA-EMC#2762)
  Fix erroneous cdump addition (NOAA-EMC#2803)
  Update ocean post-processing triggers (NOAA-EMC#2784)
  Update the gfs_utils repository hash (NOAA-EMC#2801)
  Add fixes for metplus jobs when gfs_cyc=2 or 4 (NOAA-EMC#2791)
  Simplify resource-related variables, remove CDUMP where unneeded (NOAA-EMC#2727)
  Remove f000 from atmos rocoto tasks for replay cases (NOAA-EMC#2778)
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Aug 9, 2024
@DavidHuber-NOAA
Merge remote-tracking branch 'origin/develop' into fix/c768_hera
17efba2 
* origin/develop:
  Stage atmospheric backgrounds and UFS cubed-sphere history files (NOAA-EMC#2792)
  Check that a PR driver is still running before trying to kill it (NOAA-EMC#2799)
  Feature/get arch adds an empty archive job to GEFS system (NOAA-EMC#2772)
  Marine DA updates (NOAA-EMC#2802)
  Revert MSU FIX_DIRs back to glopara (NOAA-EMC#2811)
  Bugfix for updating label states in Jenkins (NOAA-EMC#2808)
  Clean-up temporary rundirs - take 2. (NOAA-EMC#2753)
  Change land surface for HR4 (NOAA-EMC#2787)
  Run METplus serially and correct the name of prod tasks (NOAA-EMC#2804)
  Update Java Agent launching script for Jenkins connections (NOAA-EMC#2762)
  Fix erroneous cdump addition (NOAA-EMC#2803)
  Update ocean post-processing triggers (NOAA-EMC#2784)
  Update the gfs_utils repository hash (NOAA-EMC#2801)
  Add fixes for metplus jobs when gfs_cyc=2 or 4 (NOAA-EMC#2791)
  Simplify resource-related variables, remove CDUMP where unneeded (NOAA-EMC#2727)
  Remove f000 from atmos rocoto tasks for replay cases (NOAA-EMC#2778)
@RussTreadon-NOAA RussTreadon-NOAA mentioned this pull request Sep 11, 2024
Closed
@DavidNew-NOAA DavidNew-NOAA deleted the feature/cubed_sphere_bkg branch November 15, 2024 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

@RussTreadon-NOAA RussTreadon-NOAA RussTreadon-NOAA left review comments

Assignees
No one assigned
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@DavidNew-NOAA @RussTreadon-NOAA @emcbot @WalterKolczynski-NOAA @danholdaway @aerorahul @DavidHuber-NOAA