Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port testlist and PE layouts to Derecho -- cesm2.2 #2281

Merged
merged 16 commits into from
Dec 12, 2023
Merged

Port testlist and PE layouts to Derecho -- cesm2.2 #2281

merged 16 commits into from
Dec 12, 2023

Conversation

ekluzek
Copy link
Collaborator

@ekluzek ekluzek commented Dec 5, 2023 •
edited
Loading

Description of changes

Move testlist and PE layouts to Derecho.
Remove tools (point users to development
versions for this). Get run_sys_test working
on Derecho. Remove hardcoded /glade/p paths.

Specific notes

CTSM Issues Fixed (include github issue #):
Resolves #2257

Are answers expected to change (and if so in what way)? No

Any User Interface Changes (namelist or namelist defaults changes)? No

Testing performed, if any: Running all testing on Derecho

@ekluzek ekluzek added the enhancement new capability or improved behavior of existing capability label Dec 5, 2023
@ekluzek ekluzek added this to the Cheyenne shutdown milestone Dec 5, 2023
@ekluzek ekluzek self-assigned this Dec 5, 2023
@ekluzek
Copy link
Collaborator Author

ekluzek commented Dec 5, 2023

Unexpected fails that I'm getting so far are:

ERP_D.f10_f10_musgs.IHistClm50Bgc.derecho_intel.clm-decStart
ERP_D_Ld10_P36x2.f10_f10_musgs.IHistClm50BgcCrop.derecho_intel.clm-ciso_decStart
ERP_D_Ld5.f10_f10_musgs.I2000Clm50Sp.derecho_intel.clm-decStart
ERP_D_Ld5.f19_g17.I2000Clm50BgcCruGs.derecho_intel.clm-fire_emis
ERP_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput
ERP_P72x2_Lm25.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-monthly
ERP_P72x2_Lm36.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-clm50cropIrrigMonth_interp
ERP_P72x2_Lm7.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-irrig_alternate_monthly
ERP_P72x2_Ly3.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-irrig_o3_reduceOutput
ERS_Ln9.ne0ARCTICne30x4_ne0ARCTICne30x4_mt12.IHistClm50SpGs.derecho_intel.clm-clm50cam6LndTuningMode_1979Start
ERS_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCropG.derecho_intel.clm-cropMonthOutput
LILACSMOKE_Vnuopc_D_Ld2.f10_f10_musgs.I2000Ctsm50NwpSpAsRsGs.derecho_intel.clm-lilac

The restart issues (ERP and ERS) seem to be issues with MOSART. Is this maybe the issue I saw with the mosart1_0_37_1 tag?

@ekluzek ekluzek changed the title Port testlist and PE layouts to Derecho Port testlist and PE layouts to Derecho -- cesm2.2 Dec 8, 2023
@ekluzek
Copy link
Collaborator Author

ekluzek commented Dec 12, 2023

Exact restart tests failing are:

FAIL ERP_D.f10_f10_musgs.IHistClm50Bgc.derecho_intel.clm-decStart COMPARE_base_rest
FAIL ERP_D_Ld10_P36x2.f10_f10_musgs.IHistClm50BgcCrop.derecho_intel.clm-ciso_decStart COMPARE_base_rest
FAIL ERP_D_Ld5.f10_f10_musgs.I2000Clm50Sp.derecho_intel.clm-decStart COMPARE_base_rest
FAIL ERS_Ln9.ne0ARCTICne30x4_ne0ARCTICne30x4_mt12.IHistClm50SpGs.derecho_intel.clm-clm50cam6LndTuningMode_1979Start COMPARE_base_rest

With these failing at the RUN step going over the 1:40 wallclock time

FAIL ERP_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput RUN time=6044
FAIL ERP_P72x2_Lm25.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-monthly RUN time=6037
FAIL ERP_P72x2_Lm36.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-clm50cropIrrigMonth_interp RUN time=6040
FAIL ERP_P72x2_Lm7.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-irrig_alternate_monthly RUN time=1836
FAIL ERP_P72x2_Ly3.f10_f10_musgs.I2000Clm50BgcCrop.derecho_intel.clm-irrig_o3_reduceOutput RUN time=6053
FAIL ERS_Ly3_P72x2.f10_f10_musgs.IHistClm50BgcCropG.derecho_intel.clm-cropMonthOutput RUN time=6045

This one fails at the MPI_BARRIER, presumably from one task dying and then other tasks waiting for it finally dying after they timeout?

FAIL ERP_D_Ld5.f19_g17.I2000Clm50BgcCruGs.derecho_intel.clm-fire_emis RUN time=97

@ekluzek
Copy link
Collaborator Author

ekluzek commented Dec 12, 2023

It looks like the MOSART problem is just that MOSART files aren't being written out at a frequency that you can do a ERP test on them. I just need to make sure MOSART history files are output often enough to do these tests. So I just need to modify the testmods to get this to work.

@ekluzek ekluzek merged commit ce322a1 into ESCOMP:release-cesm2.2 Dec 12, 2023
1 check passed
@ekluzek ekluzek deleted the cesm22_derecho branch December 12, 2023 20:51
ekluzek added a commit to ekluzek/CTSM that referenced this pull request Jan 24, 2024
@ekluzek
Merge pull request ESCOMP#2281 from ekluzek/cesm22_derecho
677873c 
Port testlist and PE layouts to Derecho -- cesm2.2
 Conflicts:
	Externals.cfg
	bld/namelist_files/namelist_defaults_ctsm_tools.xml
	cime_config/testdefs/testlist_clm.xml
	cime_config/testdefs/testlist_clm_nuopc.xml
	doc/.ChangeLog_template
	doc/ChangeSum
	doc/release-cesm2.2.ChangeLog
	test/tools/README
 	test/tools/nl_files/mkmapdata_if10
 	test/tools/test_driver.sh
 	test/tools/tests_pretag_cheyenne_nompi
 	tools/contrib/SpinupStability.ncl
 	tools/contrib/run_clm_historical
 	tools/mkmapdata/README
 	tools/mkmapdata/mkmapdata.sh
 	tools/mkmapdata/regridbatch.sh
 	tools/mksurfdata_map/README.developers
 	tools/mksurfdata_map/mksurfdata.pl
 	tools/mksurfdata_map/mksurfdata_map.namelist
 	tools/mksurfdata_map/src/mkVICparamsMod.F90
 	tools/mksurfdata_map/src/mkagfirepkmonthMod.F90
 	tools/mksurfdata_map/src/mkdiagnosticsMod.F90
 	tools/mksurfdata_map/src/mkdomainMod.F90
 	tools/mksurfdata_map/src/mkfileMod.F90
 	tools/mksurfdata_map/src/mkgdpMod.F90
 	tools/mksurfdata_map/src/mkglacierregionMod.F90
 	tools/mksurfdata_map/src/mkglcmecMod.F90
 	tools/mksurfdata_map/src/mkgridmapMod.F90
 	tools/mksurfdata_map/src/mkharvestMod.F90
 	tools/mksurfdata_map/src/mkindexmapMod.F90
 	tools/mksurfdata_map/src/mklaiMod.F90
 	tools/mksurfdata_map/src/mklanwatMod.F90
 	tools/mksurfdata_map/src/mkpeatMod.F90
 	tools/mksurfdata_map/src/mkpftMod.F90
 	tools/mksurfdata_map/src/mksoilMod.F90
 	tools/mksurfdata_map/src/mksoilUtilsMod.F90
 	tools/mksurfdata_map/src/mksoildepthMod.F90
 	tools/mksurfdata_map/src/mksurfdat.F90
 	tools/mksurfdata_map/src/mktopostatsMod.F90
 	tools/mksurfdata_map/src/mkurbanparCommonMod.F90
 	tools/mksurfdata_map/src/mkurbanparMod.F90
 	tools/mksurfdata_map/src/mkvarctl.F90
 	tools/mksurfdata_map/src/mkvarpar.F90
 	tools/mksurfdata_map/src/mkvocefMod.F90
 	tools/mksurfdata_map/src/test/mkgridmap_test/test_mkgridmap.pf
 	tools/mksurfdata_map/src/test/mkindexmap_test/test_mkindexmap.pf
@ekluzek ekluzek mentioned this pull request Jan 24, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@ekluzek ekluzek

Labels
enhancement new capability or improved behavior of existing capability
Projects
CTSM: Cheyenne to Derecho transition
Status: Done
Milestone
Cheyenne shutdown
Development

Successfully merging this pull request may close these issues.
1 participant
@ekluzek