Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ctsm5.2.0 -- ctsm5.2.mksurfdata #2372

Merged
merged 683 commits into from
Apr 20, 2024
Merged

ctsm5.2.0 -- ctsm5.2.mksurfdata #2372

merged 683 commits into from
Apr 20, 2024

Conversation

slevis-lmwg
Copy link
Contributor

@slevis-lmwg slevis-lmwg commented Feb 16, 2024 •
edited by ekluzek
Loading

Description of changes

This pull request will document final steps in the ctsm5.2 branch development before we make the ctsm5.2.0 tag.

Specific notes

Contributors other than yourself, if any:
@mvertens @ekluzek @jedwards4b @billsacks @wwieder @lawrencepj1 @negin513 @dlawrenncar @olyson @keerzhang1 @fang-bowen @Face2sea @adamrher @samsrabin

CTSM Issues Fixed (include github issue #):
Fixes #1903
Fixes #1716
Fixes #1556
Fixes #1878
Fixes #2131
Fixes #2218
Fixes #1483
Fixes #2228
Fixes #90
Fixes #80
Fixes #1878

Pull Requests that comprise this effort:
https://github.com/ESCOMP/CTSM/pulls?q=is%3Apr+base%3Actsm5.2.mksurfdata+is%3Aclosed
#2464
#2447
#2427
#2424
#2417
#2327
#2318
#2164
#2048
#2016
#2008
#1946
#1920
#1890
#1873
#1866
#1853
#1796
#1756
#1748
#1746
#1732
#1728
#1721
#1663
#1586

Are answers expected to change (and if so in what way)?
Yes, more than roundoff for at least the following reasons:

  • Would have expected roundoff if we had just updated mksurfdata_map with mksurfdata_esmf
  • More than roundoff because we have replaced several raw datasets to include more modern data

Any User Interface Changes (namelist or namelist defaults changes)?

  • mksurfdata_esmf has a less cumbersome workflow than mksurfdata_map
  • namelist_defaults point to new fsurdat, landuse, and finidat files

Testing performed, if any:

ekluzek and others added 30 commits October 17, 2023 21:26
@ekluzek
Add tests for casper and izumi
7155d4e
@ekluzek
Add some tests on the node and task counts and restrict them to posit…
232297d 
…ive integers
@ekluzek
Remove subprocess so can call the namelist directly, test now works
c1e1e02
@ekluzek
Add test for multi jobscript
7cbe492
@ekluzek
Add black commit to list
1609ff0
@ekluzek
Get makefile working with updated script
9cb21e8
@slevis-lmwg
Introduce istocn = 3 and related code
83bc76f
@ekluzek
Get test to work with valid scenario, remove the uneeded all and stan…
d876d78 
…dard scenarios, and add error checking for resolution and scenario
@ekluzek
Default script names end with .sh
9f1a976
@ekluzek
A few more adjustments to get the list look like the supported grids …
bc3ed7c 
…list, mostly adding an ultra-hi-res option for non-crop present day
@ekluzek
More adjustments to get the list right, don't do the ne120 SSP by def…
9935218 
…ault, nor ultra-high res
@ekluzek
More adjustments to get the list right, don't do the ne120 SSP by def…
a766c28 
…ault, nor ultra-high res
@ekluzek
More work at getting the existing build or not working
2460482
@ekluzek
Fix fdyndat filename
6be1728
@slevis-lmwg
Merge suma lines that need not be separate
9ee7850
@ekluzek
add 360x720cru to standard resolutions
889c0cd
@slevis-lmwg
Remove comment that referred to past code history
0d7264a
@ekluzek
Remove the grids that won't be used, add some of the new ones, make s…
0c2c886 
…ure crop datasets are used for all options if possible
@ekluzek
Merge branch 'ctsm5.2_makefile' of github.com:ekluzek/CTSM into ctsm5…
b7b52be 
….2_makefile
@ekluzek
Merge branch 'ctsm5.2_makefile' of github.com:ekluzek/CTSM into ctsm5…
352bd12 
….2_makefile
@ekluzek
Write out number of tasks as an attribute to the output file
dbfa9fa
@ekluzek
Add more to the git ignore for files in tools/mksurfdata_esmf
f9ef5a4
@ekluzek
Reduce the number of nodes used for most cases, and make the SSP case…
61baf12 
…s do only one resolution so can fit in a reasonable time
@ekluzek
Seperate out the SSP cases by single resolution, remove some of the s…
5179d62 
…cenarios that put everything into one thing
@slevis-lmwg
Updates for ./run_ctsm_py_tests --sys and --unit to pass
c4420db
@slevis-lmwg
Add convert_ocean_to_land = .true. to MKSURFDATAESMF test for PASS
d42b32c
@slevis-lmwg
Black reformatging
a0d014f
@slevis-lmwg
Update .git-blame-ignore-revs
42e6796
@slevis-lmwg
Merge remote-tracking branch 'escomp/ctsm5.2.mksurfdata' into ocn_v_w…
00b6347 
…etland_iss1886

Resolved conflicts:
python/ctsm/test/test_sys_fsurdat_modifier.py
python/ctsm/test/testinputs/default_data.cfg
python/ctsm/test/testinputs/modify_fsurdat_short.cfg
@slevis-lmwg
Correct accidental line-delete from resolving conflicts
f532510
ekluzek and others added 17 commits April 16, 2024 16:55
@ekluzek
Add more existance tests of required scripts
8e0334e
@ekluzek
Add more error checking and get the test to work
b5667fa
@ekluzek
Add clm60 lndtuningmode for cam60 with ZDust
434edda
@ekluzek
Change test from clm51 to clm60
f0f0743
@ekluzek
Align compset wtih lnd_tuning_mode
fbe3285
@ekluzek
Merge branch 'updatetodev176' of github.com:ekluzek/CTSM into updatet…
4f541fe 
…odev176
@ekluzek
Remove debug test, and run through black
374f8b8
@ekluzek
Add expected fail to a fates test that's documented for failing
b41171a
@ekluzek
Add one of the NEON tests in the fates testsuite to expected fail for…
aca1530 
… NEON server or file length issues
@ekluzek
Another FATES expected fail
172e746
@ekluzek
Another expected fails
52bec1f
@ekluzek
Merge branch 'updatetodev176' of github.com:ekluzek/CTSM into updatet…
447d783 
…odev176
@ekluzek
Add expected fail for a MOSART test because it's using a Clm51 compse…
35404eb 
…t and needs to update to Clm60
@ekluzek
Add more notes about the testlists
6926f4b
@ekluzek
Merge branch 'updatetodev176' of github.com:ekluzek/CTSM into updatet…
ce6605c 
…odev176
@ekluzek
More expected fails because of long filenames
78b2267
@ekluzek
Merge pull request #2464 from ekluzek/updatetodev176
7746e9f 
Update ctsm5.2 branch to ctsm5.1.dev176 and some small updates
@slevis-lmwg
Copy link
Contributor Author

I submitted these izumi tests to help get around the glitches that Erik saw on izumi.
I may have misnamed the new baselines.
I added _slevis suffixes to them rather than deleting Erik's:

./run_sys_tests -s fates -c alpha-ctsm5.2.mksrf.27_ctsm5.1.dev176 -g fates-sci.1.72.2_api.34.0.0-alpha-ctsm5.2.mksrf.27_ctsm5.1.dev176_slevis
./run_sys_tests -s mosart -c alpha-ctsm5.2.mksrf.27_ctsm5.1.dev176 -g mosart1_0_49-alpha-ctsm5.2.mksrf.27_ctsm5.1.dev176_slevis

I compared to alpha-ctsm5.2.mksrf.27_ctsm5.1.dev176 ("null" test) for now. We can run against other baselines later if we have specific questions or concerns.

@slevis-lmwg
Copy link
Contributor Author

Failure not labeled EXPECTED:
ERS_D_Ld5.1x1_brazil.I2000Clm50FatesCruRsGs.izumi_nag.clm-FatesColdHydro
but a similar test in the baseline also fails and is labeled EXPECTED:
SMS_Lm3_D_Mmpi-serial.1x1_brazil.I2000Clm50FatesCruRsGs.izumi_intel.clm-FatesColdHydro
and is documented here:
#2373
NGEET/fates#1163
#2355 (comment)

@ekluzek
Copy link
Collaborator

ekluzek commented Apr 20, 2024

I see what's up with ERS_D_Ld5.1x1_brazil.I2000Clm50FatesCruRsGs.izumi_nag.clm-FatesColdHydro. It is in the expected fail list, but as a COMPARE_base_rest fail. But, #2373 has it fail at runtime. So I'll update the entry to mention #2373.

Thanks for pointing that out @slevis-lmwg

@ekluzek ekluzek merged commit 0910183 into master Apr 20, 2024
3 checks passed
@wwieder
Copy link
Contributor

wwieder commented Apr 20, 2024

Thanks for all your work on this Erik, Sam, and many more!

@ekluzek ekluzek deleted the ctsm5.2.mksurfdata branch October 3, 2024 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ekluzek ekluzek ekluzek left review comments

@mvertens mvertens mvertens left review comments

@olyson olyson olyson approved these changes

@wwieder wwieder wwieder approved these changes

Assignees

@slevis-lmwg slevis-lmwg

Labels
enhancement new capability or improved behavior of existing capability priority: high High priority to fix/merge soon, e.g., because it is a problem in important configurations size: large Large project that will take a few weeks or more testing additions or changes to tests
Projects
CTSM: Upcoming tags
Status: Done (non release/external)
LMWG: Near Term Priorities
Archived in project
Updated datasets for CTSM5.2 (ctsm5.2.mksurfdata branch and tags)
Status: Done
Milestone
ctsm5.2.0
7 participants
@slevis-lmwg @wwieder @ekluzek @billsacks @olyson @mvertens @samsrabin