Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove manage_externals and update based on ESCOMP/CESM tag 'cesm3_0_beta01' #69

Merged
merged 12 commits into from
Aug 29, 2024
Merged

Remove manage_externals and update based on ESCOMP/CESM tag 'cesm3_0_beta01' #69

merged 12 commits into from
Aug 29, 2024

Conversation

gdicker1
Copy link
Contributor

@gdicker1 gdicker1 commented Aug 21, 2024 •
edited
Loading

NOTE: this PR is currently incomplete and does not actually have git-fleximod, nor remove manage_externals yet.

This continues to catch EarthWorks up to ESCOMP/CESM development and is expected to merge after #68.

Important

This PR represents a change to how the EarthWorksOrg code will be fetched. After cloning the PR branch (or develop branch of EarthWorks repo after this merge), a user now runs bin/git-fleximod update instead of ./manage_externals/checkout_externals.

@gdicker1 gdicker1 added external Has to do with externals EW specific This has to do with EarthWorks only - files, goals, code that probably won't be wanted upstream upstream Related to the upstream repo (ESCOMP/CESM) labels Aug 21, 2024
@gdicker1 gdicker1 self-assigned this Aug 21, 2024
@gdicker1 gdicker1 changed the title Remove manage_externals and update based on ESCOMP/CESM tag 'cesm3_0_beta01' (WIP) Remove manage_externals and update based on ESCOMP/CESM tag 'cesm3_0_beta01' Aug 21, 2024
@gdicker1 gdicker1 mentioned this pull request Aug 27, 2024
Merged
@gdicker1
Copy link
Contributor Author

Sub PRs:

gdicker1 added 10 commits August 27, 2024 17:41
@gdicker1
Update cime_config to match ESCOMP/CESM cesm3_0_beta01
f71c95a
@gdicker1
Use similar externals version to cesm3_0_beta01
1af943d 
NOTE: since these changes still use manage_externals this commit may not
be able to fully checkout (because CAM and CLM/CTSM use git-fleximod).

Also:
- Rename cice6 to cice, remove cice5 external
- Remove cpl7 external
- Rename pycect external to pysect
@gdicker1
Translate Externals.cfg to .gitmodules
488da6f 
Using the cesm3_0_beta01 .gitmodules as reference.

Some entries are included from CESM, but commented out since EarthWorks
doesn't use them. These modules/externals include: ww3, rtm, mizuroute,
fms, cism, and mom.
@gdicker1
Copy bin/git-fleximod from cesm3_0_beta01
8fa4990 
This is the entry-point for using git-fleximod in another repository.
Once .gitmodules is added, running 'bin/git-fleximod update' will fetch
all needed sub-repositories.
@gdicker1
Add git-fleximod utility
6ce12ce 
Add '.lib/git-fleximod' using git-subtree with tag 'v0.7.8' from
https://github.com/ESMCI/git-fleximod.

The commits are squashed to prevent the git-fleximod history from
confusing the EarthWorks commit history. Future updates should use a
command like:

'git subtree --squash -P .lib/git-fleximod merge <some_commit-ish>'
@gdicker1
Squashed '.lib/git-fleximod/' content from commit 8eae7e3
ad32546 
git-subtree-dir: .lib/git-fleximod
git-subtree-split: 8eae7e3fb53a3c7dff7010e90cd6990ae7f2021d
@gdicker1
Remove manage_externals/
ea8cad4 
This is now replaced by git-fleximod. If a project directory isn't
populated, a user should now try './bin/git-fleximod update' to fetch
all submodules.

Also updates the .gitignore since git submodules (which git-fleximod
uses) will handle the subdirectories for externals properly.
@gdicker1
Add new checkout process to README and script to redirect users
e51682a 
Users who look at the README.md or who try to run the command
./manage_externals/checkout_externals should now get helpful information
about how to proceed with git-fleximod.
@gdicker1
Squashed '.lib/git-fleximod/' changes from 8eae7e3..8f78adc
9bf2547 
8f78adc Bump to 0.8.4
1ee334a Merge pull request EarthWorksOrg#58 from ESMCI/fix/hashid
e705f72 fix hash id and tag parsing
36b4b92 Bump to 0.8.3
654d233 Merge pull request EarthWorksOrg#57 from ESMCI/fix_test_output
0a4ea71 update the way tags are identified
6ee78d6 correct count of test fails
75a0337 Bump to 0.8.2
7136066 Merge pull request EarthWorksOrg#54 from ESMCI/fix/tag-not-found
a90de4f update to fetch tag first
a6e43fc Bump to 0.8.1
229ba03 Merge pull request EarthWorksOrg#53 from ESMCI/fix/ssh_access
e9cd2be better fix for issue with ssh access
c624b61 Bump to 0.8.0
6ccf617 Merge pull request EarthWorksOrg#51 from ESMCI/fix/recursive
53fce57 no need to make submodule objects persist
dae1c82 now working with issue EarthWorksOrg#50
ded91fd all tests passing, issue EarthWorksOrg#50 fixed
62a338b all pytests now passing
143abac tests a-d all pass
2b021fd add documentation to submodule.py
72211d2 add the new submodule class
82dd7a4 test_b_update now working
a7d8b4e all pytests passing
5322019 Bump to 0.7.9
1f07cb6 Merge pull request EarthWorksOrg#49 from ESMCI/fix/fxDONOTUSEurl
31ce71b use poetry update to update poetry.lock file
46043f0 add code of conduct (issue EarthWorksOrg#35)
0c3d683 dependabot changes and issue EarthWorksOrg#48

git-subtree-dir: .lib/git-fleximod
git-subtree-split: 8f78adcb0386cf496de377b9a92705e44f3f9a37
@gdicker1
Update git-fleximod to v0.8.4 from https://github.com/ESMCI/git-fleximod
01ec6d5 


Merge commit '9bf25476e0db6879cd8084244e417f303b75eb96' into update/cesm3_0_beta01

Command: git subtree pull --squash -P .lib/git-fleximod https://github.com/ESMCI/git-fleximod v0.8.4

This version (which is also used in cesm3_0_beta02) works as expexted
for EarthWorks - especially fetching the correct version of MPAS within
CAM when running 'bin/git-fleximod update' in a fresh clone of EW.
@gdicker1 gdicker1 force-pushed the update/cesm3_0_beta01 branch from 2ca14ea to 1d8aa97 Compare August 28, 2024 00:31
@gdicker1 gdicker1 changed the title (WIP) Remove manage_externals and update based on ESCOMP/CESM tag 'cesm3_0_beta01' Remove manage_externals and update based on ESCOMP/CESM tag 'cesm3_0_beta01' Aug 28, 2024
@gdicker1 gdicker1 marked this pull request as ready for review August 28, 2024 15:43
@gdicker1
Copy link
Contributor Author

gdicker1 commented Aug 28, 2024 •
edited
Loading

Caution

The changes in this PR cause compsets that use CLUBB physics and NVHPC compilers (e.g. CHAOS2000dev) to fail to compile. This can be fixed by moving one file within CLUBB:

# From the root of an EarthWorks clone
mv components/cam/src/physics/clubb/src/CLUBB_core/mt95.f90 \
   components/cam/src/physics/clubb/src/CLUBB_core/mt95.F90

These changes won't be necessary once a more up-to-date version of CLUBB is used. This will be accomplished by updating CAM to be based on cam6_4_019 see ESCOMP/CAM PR#1086 for some more info.

@gdicker1 gdicker1 force-pushed the update/cesm3_0_beta01 branch from 1d8aa97 to 51e036b Compare August 28, 2024 16:25
@gdicker1 gdicker1 force-pushed the update/cesm3_0_beta01 branch from 51e036b to de67d94 Compare August 28, 2024 20:01
@gdicker1
Copy link
Contributor Author

gdicker1 commented Aug 29, 2024 •
edited
Loading

Testing the ew-pr category against runs of the same test category using the ew2.3.002 tag as baseline (results of #68) revealed:

  • QPC6 with nvhpc had NLCOMP fail 1
  • CHAOS2000dev with gnu had NLCOMP and RUN fails
  • CHAOS2000dev with nvhpc, and intel had NLCOMP fail
  • F2000climoEW with nvhpc had NLCOMP fail
  • FullyCoupledEW with nvhpc had NLCOMP fail
  • FHS94 with nvhpc only had NLCOMP fail
  • FKESSLER with nvhpc only had NLCOMP fail

The failure of CHAOS2000dev with GNU compilers is odd. The failed run creates PET{proc_num}.ESMF_LogFile for each process that failed. An excerpt seems to blame some NaNs.
mediator/med_methods_mod.F90:2628 ERROR: 1 nans found in Sa_pslv

When trying to build with DEBUG=true, builds generate an internal compiler error and point to the generation of aerosol_optics_cam.o. This doesn't seem specific to EarthWorks; using the exact same aerosol_optics_cam.F90 as cam6_3_162 still generated this error (so, the __881FIX__ and __945FIX__ aren't a problem) and this build error still occurs when I try to build F2000dev with DEBUG=true using this PR branch.
gmake: *** [/glade/derecho/scratch/gdicker/ew-pr_2024Aug28152300/ewm-2.3.001-69/SMS_P128_Vnuopc.mpasa120_oQU120.CHAOS2000dev.derecho_gnu.GC.20240828_152520_zsbol5/Tools/Makefile:1009: aerosol_optics_cam.o] Error 1

However, since these changes have been approved by CESM development and GNU isn't our primary compiler, I think these changes are ready to merge.

Footnotes

  1. NLCOMP fail means some namelist(s) changed between the test and the baseline. ↩

@gdicker1
Copy link
Contributor Author

gdicker1 commented Aug 29, 2024 •
edited
Loading

@sherimickelson and @areanddee, there's a few things to note in the description and comments here. However, I think these changes are ready to merge and plan to do so this afternoon.

This PR is 2nd of 4 PRs to update EarthWorks with CESM developments:

@areanddee
Copy link
Contributor

Approved. I hope we can resolve the .f90 -> .F90 soon. That's sure to trip people up.

@gdicker1
Finalize the cesm3_0_beta01 update
f5edd34 
Use tags from merged Sub-PRs and update URLs in .gitmodules
@gdicker1 gdicker1 merged commit 0257b38 into EarthWorksOrg:develop Aug 29, 2024
This was referenced Aug 30, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sherimickelson sherimickelson sherimickelson approved these changes

@areanddee areanddee Awaiting requested review from areanddee

Assignees

@gdicker1 gdicker1

Labels
EW specific This has to do with EarthWorks only - files, goals, code that probably won't be wanted upstream external Has to do with externals upstream Related to the upstream repo (ESCOMP/CESM)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gdicker1 @areanddee @sherimickelson