Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug fix for QPFARI and QPFFFG encoding in RRFS #968

Merged
merged 10 commits into from
Jun 12, 2024
Merged

Bug fix for QPFARI and QPFFFG encoding in RRFS #968

merged 10 commits into from
Jun 12, 2024

Conversation

EricJames-NOAA
Copy link
Contributor

@EricJames-NOAA EricJames-NOAA commented Jun 10, 2024 •
edited by WenMeng-NOAA
Loading

The main changes are:

  • Bug fix in prob values denoted in QPFARI and QPFFG
  • Bug fix in labeling of run total accumulated fields in sub-hourly
  • Bug fix in calculation of vertically summed smoke emissions
  • Change the path of g2tmpl/1.12.0 on WCOSS2 to the prod side
  • Change 1-h averaged MASSDEN to PM2.5 and PM10

prob_num and tot_num_prob were not set in grib2_module.f, resulting in seemingly random values showing up for the new GRIB2 records QPFARI and QPFFFG, as noted by Hui-Ya Chuang:

< 772:4824832913:vt=2024022023:surface:11-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat
=1 parm=197:prob >1:prob fcst 72/110
772:4824832913:vt=2024022023:surface:11-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat
=1 parm=197:prob >1:prob fcst 96/110
6177c6177
< 773:4824833141:vt=2024022023:surface:0-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat=
1 parm=197:prob >1:prob fcst 72/110

773:4824833141:vt=2024022023:surface:0-12 hour acc fcst:var discipline=1 center=7 local_table=1 parmcat=
1 parm=197:prob >1:prob fcst 96/110

This small change fixes the issue, and should lead to bitwise identical results.

The change was tested for the RRFS_NA_3km system on Hera.

@EricJames-NOAA EricJames-NOAA changed the title QPFARI bugfix Bug fix for QPFARI and QPFFFG encoding in RRFS Jun 10, 2024
@WenMeng-NOAA
Copy link
Collaborator

@EricJames-NOAA Can you sync your branch with the latest commit of the branch 'release/rrfs_v1'?

This was linked to issues Jun 10, 2024
Bug in calculation of vertically summed smoke emissions #969
Closed
Bug in prob values denoted in QPFARI and QPFFFG fields in RRFS #962
Closed
Bug in labeling of run total accumulated fields in sub-hourly output. #970
Closed
@EricJames-NOAA
Copy link
Contributor Author

@WenMeng-NOAA welcome back! I hope your family are well.

Just to clarify, yes, these changes are needed for RRFSv1.

Smoke/dust folks have requested one more small change which I hope to include in this PR if that's OK.

@WenMeng-NOAA
Copy link
Collaborator

@WenMeng-NOAA welcome back! I hope your family are well.

Just to clarify, yes, these changes are needed for RRFSv1.

Smoke/dust folks have requested one more small change which I hope to include in this PR if that's OK.

@EricJames-NOAA The modification in smoke/dust product calculation is fine with me unless @MatthewPyle-NOAA has concerns.

@WenMeng-NOAA
Copy link
Collaborator

@EricJames-NOAA Can you take a look at my test results for the sub-hourly case at /home/Wen.Meng/stmp2/rrfs_2024050203 and let me know if there are any inconsistencies with your testing?

@WenMeng-NOAA
Copy link
Collaborator

@EricJames-NOAA With the fix in grib2 meta data, the QPFARI and QPFFFG in rrfs PRSLEV dataset are encoded as:

772:714248470:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:9-10 hour acc fcst:prob >1:prob fcst 0/1
773:714248695:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:10-10 hour acc fcst:prob >1:prob fcst 0/1
774:714248920:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >2:prob fcst 0/1
775:714249145:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >2:prob fcst 0/1
776:714249370:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >5:prob fcst 0/1
777:714249595:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >5:prob fcst 0/1
778:714249820:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >10:prob fcst 0/1
779:714250045:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >10:prob fcst 0/1
780:714250270:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >100:prob fcst 0/1
781:714250495:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >100:prob fcst 0/1

Please confirm the above changes.

@WenMeng-NOAA
Copy link
Collaborator

@EricJames-NOAA Can you also combine another change in your PR?
Update g2tmpl/1.12.0 location on WCOSS2 which was officially installed at the prod side:
in UPP/modulefiles/wcoss2.lua, remove line 48, and 49:

prepend_path("MODULEPATH"," /u/wen.meng/noscrub/ncep_post/g2tmpl/libs/modulefiles/compiler/intel/19.1.3.304")
load(pathJoin("g2tmpl", g2tmpl_ver))

@EricJames-NOAA
Copy link
Contributor Author

@EricJames-NOAA Can you take a look at my test results for the sub-hourly case at /home/Wen.Meng/stmp2/rrfs_2024050203 and let me know if there are any inconsistencies with your testing?

@WenMeng-NOAA my tests were on Jet on /lfs4 which is not accessible at the moment. I am recreating my tests on Hera.

@EricJames-NOAA
Copy link
Contributor Author

@EricJames-NOAA With the fix in grib2 meta data, the QPFARI and QPFFFG in rrfs PRSLEV dataset are encoded as:

772:714248470:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:9-10 hour acc fcst:prob >1:prob fcst 0/1
773:714248695:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=197:surface:10-10 hour acc fcst:prob >1:prob fcst 0/1
774:714248920:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >2:prob fcst 0/1
775:714249145:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >2:prob fcst 0/1
776:714249370:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >5:prob fcst 0/1
777:714249595:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >5:prob fcst 0/1
778:714249820:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >10:prob fcst 0/1
779:714250045:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >10:prob fcst 0/1
780:714250270:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:9-10 hour acc fcst:prob >100:prob fcst 0/1
781:714250495:d=2023062800:var discipline=1 center=7 local_table=1 parmcat=1 parm=196:surface:10-10 hour acc fcst:prob >100:prob fcst 0/1

Please confirm the above changes.

@WenMeng-NOAA those changes look correct and consistent with my results.

@WenMeng-NOAA WenMeng-NOAA linked an issue Jun 11, 2024 that may be closed by this pull request
Update UPP wcoss2 model to use recently implemented operational G2tmpl lib v1.12.0 #960
Closed
@EricJames-NOAA
Copy link
Contributor Author

@WenMeng-NOAA your results are consistent with mine. I added the smoke change requested by NCEP, so this PR is ready from my perspective.

@WenMeng-NOAA WenMeng-NOAA linked an issue Jun 11, 2024 that may be closed by this pull request
1-h avg MASSDEN fields in RRFS need to be PM2.5 and PM10 #971
Closed
@WenMeng-NOAA
Copy link
Collaborator

@EricJames-NOAA The 1-h averaged PM2.5 and PM10 in PRSLEV dataset are labeled as:

751:704042206:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <2.5e-06
752:704280574:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <1e-05

Please confirm.

@EricJames-NOAA
Copy link
Contributor Author

@EricJames-NOAA The 1-h averaged PM2.5 and PM10 in PRSLEV dataset are labeled as:

751:704042206:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <2.5e-06
752:704280574:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <1e-05

Please confirm.

@WenMeng-NOAA yes, that is correct. We have changed these fields to be PM2.5 and PM10 (including both smoke and dust), so I changed the aerosol type to "Missing". The size categorization is correct.

@WenMeng-NOAA
Copy link
Collaborator

@EricJames-NOAA The 1-h averaged PM2.5 and PM10 in PRSLEV dataset are labeled as:

751:704042206:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <2.5e-06
752:704280574:d=2023062800:MASSDEN:8 m above ground:10-11 hour ave fcst:aerosol=Missing:aerosol_size <1e-05

Please confirm.

@WenMeng-NOAA yes, that is correct. We have changed these fields to be PM2.5 and PM10 (including both smoke and dust), so I changed the aerosol type to "Missing". The size categorization is correct.

@EricJames-NOAA Thanks for clarifying.

@WenMeng-NOAA WenMeng-NOAA merged commit fc85241 into NOAA-EMC:release/rrfs_v1 Jun 12, 2024
4 of 5 checks passed
WenMeng-NOAA added a commit to WenMeng-NOAA/UPP that referenced this pull request Jun 17, 2024
@WenMeng-NOAA
Sync the RRFS hotfix NOAA-EMC#967 and NOAA-EMC#968.
bd4f4f4
This was referenced Jun 17, 2024
Merged
Closed
WenMeng-NOAA added a commit that referenced this pull request Jun 18, 2024
@WenMeng-NOAA @FernandoAndrade-NOAA
Sync the RRFS hotfix #967 and #968. (#979)
d5a7944 
* Sync the RRFS hotfix #967 and #968.

* hera RT log

* hercules RT log

---------

Co-authored-by: FernandoAndrade-NOAA <[email protected]>
This was referenced Jun 18, 2024
Closed
Closed
Closed
Closed
Closed
Closed
DavidHuber-NOAA added a commit to DavidHuber-NOAA/UPP that referenced this pull request Sep 9, 2024
@DavidHuber-NOAA
Merge remote-tracking branch 'origin/develop' into feature/ss_180
9669aba 
* origin/develop:
  Updates to support MPAS applications (RRFS and HFIP) (NOAA-EMC#1029)
  Add 6-hrly max winds at 10m to SFS (NOAA-EMC#1019)
  Update jet.lua (NOAA-EMC#1027)
  Updates to support UPP use with MPAS (NOAA-EMC#1018)
  [develop]: Turn on WARN_AS_ERROR for Doxygen Documentation Builds (NOAA-EMC#1023)
  Sample PR for testing (NOAA-EMC#1007)
  Enable template 4-49 to obtain aerosol ensemble information (NOAA-EMC#1011)
  Relocate gfs/gefs/sfs control files (NOAA-EMC#990)
  update g2tmp on aws to 1.12.0 (NOAA-EMC#996)
  [develop]: Update requirements files (NOAA-EMC#994)
  Bump zipp from 3.17.0 to 3.19.1 in /doc/UserGuide (NOAA-EMC#991)
  remove stack-python load for Gaea build (NOAA-EMC#993)
  Bump certifi from 2024.2.2 to 2024.7.4 in /doc/UserGuide (NOAA-EMC#988)
  [develop]: Add Doxygen variable definitions to modules (NOAA-EMC#965)
  Read bucket either as an integer or float (NOAA-EMC#984)
  Update Orion module file for Rocky 9 upgrade (NOAA-EMC#987)
  UPP bugfixes for inline post and g2tmpl 1.12.0 compatibility (and number concentration updates) (NOAA-EMC#974)
  Sync the RRFS hotfix NOAA-EMC#967 and NOAA-EMC#968. (NOAA-EMC#979)
  Bump urllib3 from 2.2.1 to 2.2.2 in /doc/UserGuide (NOAA-EMC#981)
  Fix for hydrostatic runs - write out Omega and HGT from model output (NOAA-EMC#959)
  Bump requests from 2.31.0 to 2.32.0 in /doc/UserGuide (NOAA-EMC#963)
  Correct rt.sh paths (NOAA-EMC#957)
  Updates to use GRIB2 template 4.9 for QPFFFG and QPFARI variables in RRFS (NOAA-EMC#929)
  Add hourly averaged smoke PM2.5 and dust PM10 (NOAA-EMC#917)
  Update Namelist documentation for all namelist options (NOAA-EMC#940)
  update gtg code revision to v.4.16.3 (NOAA-EMC#954)
  Bump jinja2 from 3.1.3 to 3.1.4 in /doc/UserGuide (NOAA-EMC#953)
  Clean up labeling for 15-min accumulated fields (NOAA-EMC#948)
  Updates to the GSL exp1 ceiling diagnostic (NOAA-EMC#947)
  Additional RT log information (NOAA-EMC#941)
  add cmake load to modulefile/gaea.lua (NOAA-EMC#945)
  RRFS updates (NOAA-EMC#938)
  add module for AWS (NOAA-EMC#934)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WenMeng-NOAA WenMeng-NOAA WenMeng-NOAA approved these changes

@MatthewPyle-NOAA MatthewPyle-NOAA Awaiting requested review from MatthewPyle-NOAA

Assignees
No one assigned
Labels
RRFS
Projects
None yet
Milestone
No milestone
2 participants
@EricJames-NOAA @WenMeng-NOAA