Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transition CCPP to ufs-s2s-model (contains #7) #64

Merged

Conversation

climbfuji
Copy link
Collaborator

@climbfuji climbfuji commented Mar 27, 2020

This PR contains and expands upon the original PR from @panll #7.

  • add CCPP to ufs-s2s-model
  • update submodule pointer for fv3atm to point to latest version for b4b identical results for the coupled model configuration (see PR Regain bit-for-bit identical results between IPD and CCPP for coupled model runs  NOAA-EMC/fv3atm#86 for details)
  • add IPD REPRO, CCPP REPRO, CCPP PROD regression tests to NEMSCompsetRun
  • try to be (more) consistent by placing the mediator restart files of the warm start runs in the RESTART directory (old tests previously put them in MEDIATOR_after_Xday, whereas the new benchmark runs put them in RESTART)
  • update existing namelist templates (not sure if they are used at all; the standard control and gfdlmp templates) so that IPD and CCPP match each other as much as possible and that they include the latest development in fv3atm
  • temporary Python script to facilitate the process of creating baselines (to be abandoned as soon as possible when the entire process is revamped)

linlin-pan and others added 6 commits December 2, 2019 21:43
compsets/all_ccpp.input
compsets/cpld_fv3_ccpp_mom6_cice_2d_atm_flux.input
compsets/cpld_fv3_ccpp_mom6_cice_cold_atm_flux.input
compsets/fv3mom6cice5_ccpp.input
coupledFV3_CCPP_MOM6_CICE.appBuilder
@climbfuji climbfuji marked this pull request as ready for review March 30, 2020 01:57
… (Regain bit-for-bit identical results between IPD and CCPP for coupled model runs)
@@ -61,7 +61,7 @@ build fv3_mom6_cice.exe {

# The NEMSAppBuilder is another embedded bash script in nemscommon.input
build=NEMSAppBuilder(NEMS.x="@[target]",modules.nems="@[modules.nems]",
OPTS="app=coupledFV3_MOM6_CICE",md5sum="@[md5sum]")
OPTS="app=coupledFV3_MOM6_CICE rebuild",md5sum="@[md5sum]")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is "rebuild" added here? In https://github.com/ufs-community/ufs-s2s-model/blob/develop/compsets/nemscommon.input#L34 we have that all builds have to "rebuild"? This was added in the last commit because when adding a build with WW3, it was required that we rebuild always or we were having issues.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. I think this was by coincidence after creating the CCPP versions and then trying to make them look the same. I can revert this change (and make the same change on the CCPP side).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the "rebuild" can then be removed from the other build commands as well, or, given that it is in nemscommon.input?

@junwang-noaa
Copy link
Collaborator

junwang-noaa commented Mar 30, 2020 via email

@climbfuji
Copy link
Collaborator Author

Jessica, Does this mean we have to have a clean build every time when we build the model? This could cause lots of inconvenience for debugging. So if this is the case, can we have a separate build for fv3-mom6-cice5-wave for 4 components test and an fv3-mom6-cice5 executable for 3 component tests until the issue with rebuild fv3-mom6-cice5-wave is resolved?

Commit d179cc3 updates the baseline on hera (revert to new official baseline dir) and removes the redundant "rebuild" flags in the individual app builder sections.

@SMoorthi-emc
Copy link
Collaborator

SMoorthi-emc commented Mar 30, 2020 via email

@DeniseWorthen
Copy link
Collaborator

So @junwang-noaa, you're suggesting we should remove the default rebuild setting from nemscommon.input and re-insert it to the individual app build files but only for the 4-component cases. I would agree w/ that approach.

@climbfuji
Copy link
Collaborator Author

Using compile.sh is way simpler!

Tell me ...

@JessicaMeixner-NOAA
Copy link
Collaborator

@junwang-noaa Yes, we have to have a clean build every time when we build the
model for the regression tests? For debugging, I've been able to turn this off, but when making/running the regression tests you have to keep this on (like in ufs-weather-model), which is an inconvenience. We could use the 4 component exe for the 3 component tests in the future to cut down on the number of builds.

@climbfuji
Copy link
Collaborator Author

@junwang-noaa Yes, we have to have a clean build every time when we build the
model for the regression tests? For debugging, I've been able to turn this off, but when making/running the regression tests you have to keep this on (like in ufs-weather-model), which is an inconvenience. We could use the 4 component exe for the 3 component tests in the future to cut down on the number of builds.

I suggest no further trials and experiments with this PR, except for running a final round of regression tests after updating the baseline location and finishing the code review. It will be way cheaper and quicker to experiment with this after removing the IPD PROD, IPD REPRO and CCPP REPRO tests.

@SMoorthi-emc
Copy link
Collaborator

SMoorthi-emc commented Mar 30, 2020 via email

@junwang-noaa
Copy link
Collaborator

junwang-noaa commented Mar 30, 2020 via email

@DeniseWorthen
Copy link
Collaborator

Could I request that we not move the conversation off the topic of this PR for now?

@SMoorthi-emc
Copy link
Collaborator

SMoorthi-emc commented Mar 30, 2020 via email

@junwang-noaa
Copy link
Collaborator

junwang-noaa commented Mar 30, 2020 via email

@JessicaMeixner-NOAA
Copy link
Collaborator

@junwang-noaa if you use the "rebuild" option everything works fine. If you do not use the "rebuild" option we are still having issues when using WW3 and I have not tracked down that issue or resolved it yet.

@junwang-noaa
Copy link
Collaborator

junwang-noaa commented Mar 30, 2020 via email

@JessicaMeixner-NOAA
Copy link
Collaborator

I am confused, are you saying we should build with "rebuild" for fv3-mom6-cice5-ww3 exe? Currently in Dom's code: target="@[plat%EXECrt]/fv3_mom6_cice_ww3.exe" modules.nems="@[plat%INCrt]/modules_fv3_mom6_cice_ww3" md5sum="@[target].md5" # The NEMSAppBuilder is another embedded bash script in nemscommon.input build=NEMSAppBuilder(NEMS.x="@[target]",modules.nems="@[modules.nems]", OPTS="app=coupledFV3_MOM6_CICE_WW3",md5sum="@[md5sum]") how does this build (without rebuild) impact the related tests with or without ww3?

@junwang-noaa Since we have "rebuild" as the default option in compsets/nemscommon.input we do not need to specify build or rebuild in build sections in compsets/fv3mom6cice5.input But if we do specify them there, then it should be there for both the fv3mom6cice5.exe and the fv3mom6cice5ww3.exe

@junwang-noaa
Copy link
Collaborator

junwang-noaa commented Mar 30, 2020 via email

Copy link
Collaborator

@DeniseWorthen DeniseWorthen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for all the effort to create multiple RTs.

compsets/all.input Show resolved Hide resolved
@climbfuji
Copy link
Collaborator Author

@DeniseWorthen thanks for checking the PR thoroughly, in particular the issue about defining cpl_defaults twice. I am just about to finish the full suite of tests after I made the changes for the "rebuild" and updated the regression test baseline to point back to the official one.

@climbfuji
Copy link
Collaborator Author

Regression tests on hera.intel using NEMSCompsetRun all passed (after removing the redundant "rebuild" flags and changing the regression test baseline back to the official location, i.e. for the code as it is now in the PR). Logs updated in the repository.

NEMSCompsetRun.log

This PR is ready to merge from my side.

@junwang-noaa
Copy link
Collaborator

junwang-noaa commented Mar 31, 2020 via email

@climbfuji
Copy link
Collaborator Author

Thank you all! Let's wait for the 8 long run cases to finish, if there is no problem there, I think we can commit the code.

@JessicaMeixner-NOAA if your runs are stuck in the queue I can help, just need your instructions or copy the already set up directories.

…component and need to remove %DEBUG=Y decorator from FV3 or FV3_MAKEOPT is ignored
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants