Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New Mesh: Add ARRM10to60E2r1 mesh #414

Merged
merged 7 commits into from
Sep 14, 2022
Merged

New Mesh: Add ARRM10to60E2r1 mesh #414

merged 7 commits into from
Sep 14, 2022

Conversation

xylar
Copy link
Collaborator

@xylar xylar commented Jun 24, 2022

The mesh resolution has been ported from legacy COMPASS.

@xylar xylar added new mesh An E3SM mesh for special review ocean in progress This PR is not ready for review or merging labels Jun 24, 2022
@xylar xylar self-assigned this Jun 24, 2022
@pep8speaks
Copy link

pep8speaks commented Jun 24, 2022 •
edited
Loading

Hello @xylar! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 83:80: E501 line too long (84 > 79 characters)

Comment last updated at 2022-08-30 15:12:58 UTC

@xylar xylar force-pushed the port_arm10to60_mesh branch 2 times, most recently from a42b390 to da146bd Compare June 24, 2022 11:22
@xylar
Copy link
Collaborator Author

xylar commented Jun 24, 2022

@darincomeau and @milenaveneziani, if you can please figure out the missing pieces while I'm on vacation next week, I can finish this PR up when I get back to work on July 4th (not a holiday for me).

I'm still working on building the mesh but it should be done soon(ish). Here is the resolution map and construction history:

cellWidthGlobal

mesh_construction

I don't plan to try to create the initial condition run any of the other tests cases until we resolve the questions above.

@xylar
Copy link
Collaborator Author

xylar commented Jun 24, 2022 •
edited
Loading

It looks as if critical passages are off by default: https://github.com/MPAS-Dev/compass/blob/master/compass/ocean/tests/global_ocean/mesh/cull.py#L15
We need to turn them on, so that's something I will add to this PR. I think we haven't noticed this default because we haven't used the new compass to produce any official E3SM meshes yet (just for testing).

Update: What I said is not true. Critical passages are always on: https://github.com/MPAS-Dev/compass/blob/master/compass/ocean/tests/global_ocean/mesh/mesh.py#L90

@xylar
Copy link
Collaborator Author

xylar commented Jun 24, 2022

Here's culled mesh in Paraview:
arm10to60_res

@darincomeau
Copy link
Collaborator

@xylar many thanks for getting this started! @milenaveneziani and I are also on vacation next week, but I was able to address some of the comments. Would you prefer I push to this branch, or leave it to you based on the comments?

@xylar
Copy link
Collaborator Author

xylar commented Jun 24, 2022

@milenaveneziani and I are also on vacation next week

Ah, okay. Then just keep the comments coming today and we'll pick up in a week.

Would you prefer I push to this branch, or leave it to you based on the comments?

I think having you comment here and me change things is best for now. I imagine the dynamics adjustment will take a bit of trial and error. Probably it would make sense for whoever does that work to then push the final result to the branch at the end.

xylar
xylar commented Jun 24, 2022
View reviewed changes
compass/ocean/tests/global_ocean/mesh/arm10to60/dynamic_adjustment/__init__.py Outdated
Comment on lines 39 to 42
restart_times = ['0001-01-01_06:00:00', '0001-01-01_12:00:00',
'0001-01-02_00:00:00', '0001-01-03_00:00:00',
'0001-01-04_00:00:00', '0001-01-07_00:00:00',
'0001-01-31_00:00:00']
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@darincomeau and @milenaveneziani, as a starting point, I would recommend doing a month of integration like we did for WC14. As is hopefully clear here, we rand for 6 hours, 6 hours, 12 hours, 1 day, 1 day, 3 days and finally 24 days.

xylar
xylar commented Jun 24, 2022
View reviewed changes
compass/ocean/tests/global_ocean/mesh/arm10to60/dynamic_adjustment/__init__.py Outdated
Comment on lines 67 to 70
'config_dt': "'00:00:30'",
'config_btr_dt': "'00:00:01.5'",
'config_Rayleigh_friction': '.true.',
'config_Rayleigh_damping_coeff': '1.0e-3'}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For each step (except simulation), these are the values you'll want to adjust. We could try the same as the WC14 to begin with but likely the time steps will need to be reduced because of the difference in minimum resolution.

Run each damped_adjustment step in compass on its own (rather than trying to run this as a suite). Monitor the global CFL number and the kinetic energy. If the CFL gets much over 0.1, you're in trouble. If the KE spikes, you're in trouble. Some combination of a shorter time step and more Rayleigh damping typically helps.

@xylar xylar force-pushed the port_arm10to60_mesh branch 2 times, most recently from 1bbd230 to e0c754b Compare June 27, 2022 10:02
@xylar
Copy link
Collaborator Author

xylar commented Jun 27, 2022

@milenaveneziani, I switched from ARM to ARRM and to the 80-layer vertical grid, as you requested. I'm in the car so don't have time or battery life to try this out yet. Maybe give it a try yourself if you have time.

@xylar xylar force-pushed the port_arm10to60_mesh branch from e0c754b to f1de868 Compare June 27, 2022 13:06
@xylar
Copy link
Collaborator Author

xylar commented Aug 20, 2022

@milenaveneziani, I don't think one more layer will make an appreciable difference based on past experience. We just need to fix Redi so this doesn't happen.

@milenaveneziani
Copy link

ok, let's leave it at 15 then.

@xylar xylar force-pushed the port_arm10to60_mesh branch from bb1d7ad to 4e64c3e Compare August 30, 2022 15:12
@xylar xylar removed the in progress This PR is not ready for review or merging label Aug 30, 2022
@xylar xylar marked this pull request as ready for review August 30, 2022 15:13
@xylar
Copy link
Collaborator Author

xylar commented Aug 30, 2022

@milenaveneziani, @darincomeau and everyone, I am opening this back up for review after #429 just got merged. I'm happy to run this mesh through its paces again but wanted to make any changes you all might want before doing that.

@darincomeau
Copy link
Collaborator

Thanks @xylar. To update on the B-case, I'm almost at 20 years (3 months to go) on the Cori run, at which point I'm hoping we'll have enough to be able to approve the mesh, at which point I'll open the E3SM PR. I realize previously we did 50 years, but I'm hoping 20 years will be sufficient. The throughput on Cori has been incredibly slow; as to the Anvil suggestion, I was a bit wary of running a simulation there since this is primarily for a different project, especially given how busy that machine has been lately, and we'd still probably get only about 2 SYPD on a medium size layout.

@xylar
Copy link
Collaborator Author

xylar commented Sep 1, 2022

@darincomeau, thanks for the update. I can see the challenges with Anvil, though it sounds like 2 SYPD would still be an improvement on what you're getting on Cori.

@darincomeau
Copy link
Collaborator

Here is the MPAS-Analysis for years 11-20 of the B-case (finally)

milenaveneziani
milenaveneziani approved these changes Sep 6, 2022
View reviewed changes
Copy link

@milenaveneziani milenaveneziani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good to go.

@darincomeau darincomeau mentioned this pull request Sep 7, 2022
Merged
@darincomeau
Copy link
Collaborator

Linking the E3SM PR: E3SM-Project/E3SM#5178

@darincomeau
Copy link
Collaborator

@xylar here is the path for the ocn grid/IC file we ended up using for E3SM-Project/E3SM#5178

/lcrc/group/e3sm/ac.xylar/compass_1.2/anvil/test_20220730/ec_arrm_for_e3sm/ocean/global_ocean/ARRM10to60/

@xylar
Copy link
Collaborator Author

xylar commented Sep 13, 2022

Thanks @darincomeau. I created a permanent branch (https://github.com/MPAS-Dev/compass/tree/ARRM10to60E2r1) and tag (https://github.com/MPAS-Dev/compass/tree/mesh_ARRM10to60E2r1) for the version of this mesh you used in E3SM-Project/E3SM#5178. I'll review the E3SM PR as soon as I can.

darincomeau
darincomeau approved these changes Sep 14, 2022
View reviewed changes
Copy link
Collaborator

@darincomeau darincomeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xylar thanks for all this! (Sorry I meant to do this yesterday).

@xylar
Copy link
Collaborator Author

xylar commented Sep 14, 2022

Thanks! merging...

@xylar xylar merged commit eba4f96 into MPAS-Dev:master Sep 14, 2022
@xylar xylar deleted the port_arm10to60_mesh branch September 14, 2022 17:42
jonbob added a commit to E3SM-Project/E3SM that referenced this pull request Oct 3, 2022
@jonbob
Merge branch 'darincomeau/grids/add-ARRM10to60E2r1' into next (PR #5178)
a2857a4 
Add script support for ARRM10to60E2r1 ocn/ice grid

Brings in support for the ARRM10to60E2r1 ocn/ice mesh.
MPAS-Dev/compass#414 has details on the mesh generation, which is an
updated version of the oARRM60to10 mesh. This PR includes all necessary
mapping, runoff, and domain files for the following configurations:
* ne30pg2_ARRM10to60E2r1
* arcticx4v1pg2_ARRM10to60E2r1
* TL319_ARRM10to60E2r1 (JRA-forced)
* T62_ARRM10to60E2r1 (CORE-forced)

[BFB] for all currently tested configurations
jonbob added a commit to E3SM-Project/E3SM that referenced this pull request Oct 4, 2022
@jonbob
Merge branch 'darincomeau/grids/add-ARRM10to60E2r1' (PR #5178)
4aad3b2 
Add script support for ARRM10to60E2r1 ocn/ice grid

Brings in support for the ARRM10to60E2r1 ocn/ice mesh.
MPAS-Dev/compass#414 has details on the mesh generation, which is an
updated version of the oARRM60to10 mesh. This PR includes all necessary
mapping, runoff, and domain files for the following configurations:
* ne30pg2_ARRM10to60E2r1
* arcticx4v1pg2_ARRM10to60E2r1
* TL319_ARRM10to60E2r1 (JRA-forced)
* T62_ARRM10to60E2r1 (CORE-forced)

[BFB] for all currently tested configurations
@mark-petersen mark-petersen mentioned this pull request Nov 7, 2022
Closed
@milenaveneziani milenaveneziani mentioned this pull request Sep 1, 2023
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@milenaveneziani milenaveneziani milenaveneziani approved these changes

@darincomeau darincomeau darincomeau approved these changes

Assignees

@xylar xylar

Labels
new mesh An E3SM mesh for special review ocean
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@xylar @pep8speaks @darincomeau @milenaveneziani @jonbob @mark-petersen