NCEP GFSv16/GEFSv12 #140

ajhenrique · 2020-01-10T20:21:02Z

Changes including new features and bugfixes for adding WW3 to the GEFSv12 and GFSv16 coupled systems at NCEP:

Added option to write single output (gridded and point) files at individual specified output time steps (Add option to create output files at every output time step #141 ), to allow inline post-processing;
Added option to write second stream of restart files (Adding a checkpoint restart capability #1 ) with write stride different to first (default) restart file,
Updated ww3_grib for grib2 encoding following recent additions/changes in WMO tables,
Several bugfixes to new output and restart options, including alignment of MPI calls and resources, backwards compatibility with multiple output configurations specified in regtests,
Changes to compile options for WCOSS to allow b4b.
Please find results from the regtest matrix comparisons in the comment below.

This reverts commit c471cda.

…jhenrique:/WW3/develop

FB_gefsv12: Adding changes to adjust to NCEP operations. * Changed compile option for NCEP Dell Phase 3, * Removed empty line on w3iogomd, * Expanded output parameters to include Tz, depth, mean direction on ww3_grib, * Enforced NOSW=3 on ww3_grid for NCEP operations (NCO switch). * Added cmplr.env support to new NCEP Dell P3 comp/link * Changed opt in cmplr.env, comp/link.wcoss_dell_p3 following suggestion to consolidate compiler flags eliminating redundancies and generalizing blocks for several compiler/machine configurations.

…smfmd.ftn

On lines 1439 and 1440, the calculations for ABX2 and ABY2 were like that of ABX and ABY. This may have been an oversight when inserted. This PR fixes the issue and aligns the code with parameter definitions in the manual.

… scripts.

…ormance slightly, which can be welcome in operational applications.

…ency FR1, temporarily provides fix for grib2 encoding failing due to unrealistically low peak frequency values

… to lowest discrete frequency FR1, temporarily provides fix for grib2 encoding failing due to unrealistically low peak frequency values

…tarting off with changes to out_grd and out_pnt to complete checkpointg with restart files at checkpointing stride.

…int output to single file per output stride. Cleaned up auxiliary parameters. Added single output file marker defined as YYYYMMDD.HHMMSS prefix.

…w not an actual output type but an add-on to output type 4.

…plr.env intel option (which are b4b). Adjustments to w3wavemd to correct log file column width changed by mistake.

…d caused compile error in the coupled system.

Issue with infinite loop reading individual grid output settings in multi.inp solved (#138)

…hould not be active in this regtest (only in mww3_test_04)

… at NCEP wcoss dell.

… reinstating log.mww3 column width as new restart stream will be reported jointly with first restart file productio.

ajhenrique · 2020-01-10T20:29:06Z

Results from regtest output compared to develop.
matrix.zip

JessicaMeixner-NOAA

Hi - A few questions:

Documentation in the manual is missing.
Update of generic input files in model/inp and model/nml are missing
Are the regression test changes testing these new features? Which regression tests can I run to test which new features? Were they added to matrix.base?
I know that check point restarting has this issue: Adding a checkpoint restart capability #1 is there an associated issue with the inline post updates? Maybe that can help answer questions like with the inline post option are you directly getting netcdf, grib, files generated or instead is it just a single file combining both the gridded and point output written for each time step?

model/bin/switch_NCEP_gwm

model/ftn/w3initmd.ftn

model/ftn/w3iogomd.ftn

model/ftn/w3wavemd.ftn

model/ftn/wmesmfmd.ftn

JessicaMeixner-NOAA · 2020-01-10T21:35:23Z

model/ftn/wmesmfmd.ftn

+
+!      call ESMF_ClockGet(extClock, startTime=ttmp, rc=rc)
+!      if (ESMF_LogFoundError(rc, PASSTHRU)) return
+      call ESMF_ClockGet(extClock, currTime=ttmp, rc=rc)


To me this does not match the email thread from @ajhenrique "Tim and I agreed that for now we should use the "if currTime > startTime" approach: in InitializeP1, get startTime, currTime, and stopTime from the external clock. If currTime > startTime, then set WW3 stime from currTime, otherwise set WW3 stime from startTime."

@JessicaMeixner-NOAA thanks again for noticing, these changes were made in a separate machine and not pushed to the repo. This will be completed shortly.

ajhenrique · 2020-01-10T21:57:19Z

Hi - A few questions:

1. Documentation in the manual is missing.

2. Update of generic input files in model/inp and model/nml are missing

3. Are the regression test changes testing these new features? Which regression tests can I run to test which new features?  Were they added to matrix.base?

4. I know that check point restarting has this issue: #1 is there an associated issue with the inline post updates?  Maybe that can help answer questions like with the inline post option are you directly getting netcdf, grib, files generated or instead is it just a single file combining both the gridded and point output written for each time step?

@JessicaMeixner-NOAA I have @RobertoPadilla-NOAA helping me to resolve these missing bits and pieces. For now, the integration of the PR would be required to move ahead with the coupled ufs-weather-model update which is needed for the upcoming GFS and GEFS upgrades. The issues you outline have been included in issue #142 . (In the meantime, if you would like to run the new features, you can use grid sets grdset_d_r for additional restarts and grdset_d_h for single-file-per-time-step output in mww3_test_06 .)

Reinstated error block for time checking in subroutine ModelAdvance

…ew features output files at every output time step and second stream of restart files.

JessicaMeixner-NOAA · 2020-01-13T15:31:10Z

Based on the original description
"Added option to write single output (gridded and point) files at individual specified output time steps (#141 ), to allow inline post-processing;"

After running:
./bin/run_test -b slurm -c hera -m grdset_d_h -s PR3_UQ_SCRIP -w work_h ../model mww3_test_06
There is not 1 single file per time step for each of the gridded and point outputs, so:

19680606.000000.out_grd.arcticsub
19680606.000000.out_grd.gbandd360
19680606.000000.out_pnt.points
19680606.010000.out_grd.arcticsub
19680606.010000.out_grd.gbandd360
19680606.010000.out_pnt.points
19680606.020000.out_grd.arcticsub
19680606.020000.out_grd.gbandd360
19680606.020000.out_pnt.points
....

The comment also says this is for "inline post processing" however, there does not appear to be any additional post processing inline in the code. This is simply splitting the output files per-time step, which is needed for checkpoint restarting. I believe the description is misleading here. Lastly, the output programs (or the regression test system) do not appear to be updated for this new splitting of the output binary files as no output is produced during the regression test.

ajhenrique · 2020-01-13T15:56:37Z

Based on the original description
"Added option to write single output (gridded and point) files at individual specified output time steps (#141 ), to allow inline post-processing;"
* After running:
  `./bin/run_test -b slurm -c hera -m grdset_d_h  -s PR3_UQ_SCRIP -w work_h ../model mww3_test_06`
  There is not 1 single file per time step for each of the gridded and point outputs, so:
19680606.000000.out_grd.arcticsub
19680606.000000.out_grd.gbandd360
19680606.000000.out_pnt.points
19680606.010000.out_grd.arcticsub
19680606.010000.out_grd.gbandd360
19680606.010000.out_pnt.points
19680606.020000.out_grd.arcticsub
19680606.020000.out_grd.gbandd360
19680606.020000.out_pnt.points
.... 
The comment also says this is for "inline post processing" however, there does not appear to be any additional post processing inline in the code. This is simply splitting the output files per-time step, which is needed for checkpoint restarting. I believe the description is misleading here. Lastly, the output programs (or the regression test system) do not appear to be updated for this new splitting of the output binary files as no output is produced during the regression test.

Hi @JessicaMeixner-NOAA I don't understand your issue. When running the regtest as stated, on Hera and WCOSS, I get the expected output of a single file for each grid and each output time step as in:

-bash-4.2$ ls -l mww3_test_06/work_h
total 21568
-rw-r--r-- 1 Henrique.Alves ocean 154246 Jan 13 15:48 19680606.000000.out_grd.arcticsub
-rw-r--r-- 1 Henrique.Alves ocean 85906 Jan 13 15:48 19680606.000000.out_grd.gbandd360
-rw-r--r-- 1 Henrique.Alves ocean 6097 Jan 13 15:48 19680606.000000.out_pnt.points
-rw-r--r-- 1 Henrique.Alves ocean 154246 Jan 13 15:48 19680606.010000.out_grd.arcticsub
-rw-r--r-- 1 Henrique.Alves ocean 85906 Jan 13 15:48 19680606.010000.out_grd.gbandd360
-rw-r--r-- 1 Henrique.Alves ocean 6097 Jan 13 15:48 19680606.010000.out_pnt.points
-rw-r--r-- 1 Henrique.Alves ocean 154246 Jan 13 15:48 19680606.020000.out_grd.arcticsub
-rw-r--r-- 1 Henrique.Alves ocean 85906 Jan 13 15:48 19680606.020000.out_grd.gbandd360
-rw-r--r-- 1 Henrique.Alves ocean 6097 Jan 13 15:48 19680606.020000.out_pnt.points
-rw-r--r-- 1 Henrique.Alves ocean 154246 Jan 13 15:48 19680606.030000.out_grd.arcticsub
-rw-r--r-- 1 Henrique.Alves ocean 85906 Jan 13 15:48 19680606.030000.out_grd.gbandd360
-rw-r--r-- 1 Henrique.Alves ocean 6097 Jan 13 15:48 19680606.030000.out_pnt.points
-rw-r--r-- 1 Henrique.Alves ocean 154246 Jan 13 15:48 19680606.040000.out_grd.arcticsub
-rw-r--r-- 1 Henrique.Alves ocean 85906 Jan 13 15:48 19680606.040000.out_grd.gbandd360
-rw-r--r-- 1 Henrique.Alves ocean 6097 Jan 13 15:48 19680606.040000.out_pnt.points

Also, as discussed in person and by email, the reference to "inline post" was erroneously used and will be updated.

JessicaMeixner-NOAA

Since it has been clarified offline that no process from the ww3_o* are intended to be merged into the separation of the output files by time stamps and that generating ascii or netcdf from the separated output files will be added in the following PR, this is fine.

ajhenrique · 2020-01-13T17:39:16Z

Since it has been clarified offline that no process from the ww3_o* are intended to be merged into the separation of the output files by time stamps and that generating ascii or netcdf from the separated output files will be added in the following PR, this is fine.

Thank you @JessicaMeixner-NOAA !

ajhenrique and others added 30 commits June 10, 2019 15:17

Updating to latest upstream (NOAA-EMC/WW3).

c471cda

Revert "Updating to latest upstream (NOAA-EMC/WW3)."

5acd9d2

This reverts commit c471cda.

Merge remote-tracking branch 'upstream/develop' into develop

a939155

Merge remote-tracking branch 'upstream/develop' into develop. 20190625

a6cf904

Merge remote-tracking branch 'upstream/develop' commit 4bacd02 into a…

818a52f

…jhenrique:/WW3/develop

Merge remote-tracking branch 'upstream/develop' into develop

951aafa

Fixing indices nx/ny for exporting ESMF grid in module /model/ftn/wme…

9a5f433

…smfmd.ftn

Correcting the ABX2/ABY2 computation in w3iogomd.ftn

25e7c42

On lines 1439 and 1440, the calculations for ABX2 and ABY2 were like that of ABX and ABY. This may have been an oversight when inserted. This PR fixes the issue and aligns the code with parameter definitions in the manual.

Replacing single for double quotes to correct comp/link.wcoss_dell_p3…

da0942c

… scripts.

Added output for fields in separated files every delta t for output

bb8c07d

Moving profiling flag -p to optional block. Removing -p improves perf…

fcd5a6d

…ormance slightly, which can be welcome in operational applications.

Merge branch 'develop' into GEFS_v12

9c44742

working for field ouputs

91bf936

multi and shel are working for inline field ouputs

ebdea19

progressing in inline point ouputs

1f03713

Adding lower bound for frequency value equal to lowest discrete frequ…

dddb378

…ency FR1, temporarily provides fix for grib2 encoding failing due to unrealistically low peak frequency values

Adding to w3iogomd and ww3_grib lower bound for frequency value equal…

f7acb17

… to lowest discrete frequency FR1, temporarily provides fix for grib2 encoding failing due to unrealistically low peak frequency values

InlinePost ready for namelist as well

0a220ff

Working for shel and multi

6c4c82f

inline posproc ready for namelist as well

7136168

Cleaning the code

1b9c448

Inline postproc working w/o the parameter

a5f4807

merging to upstream, solving conflic on w3iogomd.ftn

5586598

Changing the output file names

65aca01

changing the naming of output files

e41efa0

Merge remote-tracking branch 'upstream/develop' into develop

76bb054

Merge remote-tracking branch 'RP/InLinePosproc' into FB_checkpoint: s…

956c940

…tarting off with changes to out_grd and out_pnt to complete checkpointg with restart files at checkpointing stride.

Merge remote-tracking branch 'origin/develop' into GEFS_v12

5a37118

Reverting changes adding FCSTTIME counter for optional gridded and po…

d499cee

…int output to single file per output stride. Cleaned up auxiliary parameters. Added single output file marker defined as YYYYMMDD.HHMMSS prefix.

ajhenrique and others added 10 commits January 6, 2020 14:51

Adding FLOUT(8) to MPI array allocation: this additional output is no…

e7f9bb2

…w not an actual output type but an add-on to output type 4.

Merge branch 'GEFS_v12' of github.com:NOAA-EMC/WW3 into GEFS_v12

d214296

Changes to comp/link flags to attempt b4b replicating results from cm…

e651159

…plr.env intel option (which are b4b). Adjustments to w3wavemd to correct log file column width changed by mistake.

Changing back ODAT size to 35 as this was mistakenly changed to 40 an…

5b0057c

…d caused compile error in the coupled system.

Bugfix to second restart file generation

ba922c3

Issue with infinite loop reading individual grid output settings in multi.inp solved (#138)

Changing shel.inp to previous version: testing of new inline output s…

f2b739c

…hould not be active in this regtest (only in mww3_test_04)

Changing optimization level to avoid memalloc crash in coupled system…

d6bc44b

… at NCEP wcoss dell.

Merge branch 'GEFS_v12' of https://github.com/NOAA-EMC/WW3 into GEFS_v12

5242ff0

Cleaning up unnecessary messages, removing lower FP bound constraint,…

97dd0e3

… reinstating log.mww3 column width as new restart stream will be reported jointly with first restart file productio.

Merge branch 'GEFS_v12' of github.com:NOAA-EMC/WW3 into GEFS_v12

7d3c550

ajhenrique requested review from aliabdolali and JessicaMeixner-NOAA January 10, 2020 20:30

aliabdolali approved these changes Jan 10, 2020

View reviewed changes

JessicaMeixner-NOAA reviewed Jan 10, 2020

View reviewed changes

ajhenrique mentioned this pull request Jan 10, 2020

Update documentation and regtest matrix for new features in issues #1 and #141 #142

Open

ajhenrique and others added 7 commits January 10, 2020 17:10

Update switch_NCEP_gwm

1b18c42

Update w3initmd.ftn

027aba2

Update w3iogomd.ftn

56f889f

Update w3wavemd.ftn

3e600b4

Update wmesmfmd.ftn

dfcf146

Reinstated error block for time checking in subroutine ModelAdvance

Uploading changes to wmesmfmd and updating comp/link options for Dell

b1d780b

Expanding info file to provide examples on how to run regtests with n…

081a8c7

…ew features output files at every output time step and second stream of restart files.

JessicaMeixner-NOAA approved these changes Jan 13, 2020

View reviewed changes

ajhenrique merged commit 328a42a into develop Jan 13, 2020

mickaelaccensi mentioned this pull request Sep 8, 2023

Bugfix/nml #1070

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NCEP GFSv16/GEFSv12 #140

NCEP GFSv16/GEFSv12 #140

ajhenrique commented Jan 10, 2020 •

edited

Loading

ajhenrique commented Jan 10, 2020

JessicaMeixner-NOAA left a comment

JessicaMeixner-NOAA Jan 10, 2020

ajhenrique Jan 10, 2020

ajhenrique commented Jan 10, 2020 •

edited

Loading

JessicaMeixner-NOAA commented Jan 13, 2020

ajhenrique commented Jan 13, 2020

JessicaMeixner-NOAA left a comment

ajhenrique commented Jan 13, 2020

NCEP GFSv16/GEFSv12 #140

NCEP GFSv16/GEFSv12 #140

Conversation

ajhenrique commented Jan 10, 2020 • edited Loading

ajhenrique commented Jan 10, 2020

JessicaMeixner-NOAA left a comment

Choose a reason for hiding this comment

JessicaMeixner-NOAA Jan 10, 2020

Choose a reason for hiding this comment

ajhenrique Jan 10, 2020

Choose a reason for hiding this comment

ajhenrique commented Jan 10, 2020 • edited Loading

JessicaMeixner-NOAA commented Jan 13, 2020

ajhenrique commented Jan 13, 2020

JessicaMeixner-NOAA left a comment

Choose a reason for hiding this comment

ajhenrique commented Jan 13, 2020

ajhenrique commented Jan 10, 2020 •

edited

Loading

ajhenrique commented Jan 10, 2020 •

edited

Loading