Microphysics package with saturation adjustment #522

OngChia · 2024-08-13T11:35:21Z

Add saturation adjustment component to icon4py. All future physics will be placed under subgrid_scale_phyiscs folder.

Saturation adjustment component has a run function that returns the tendencies, conforming with the decision made in this PR Component interface #506. However, a follow-up action needs to be done in a subsequent PR to make this component fully follow the protocol.
A datatest is added to test the port of saturation adjustment using the serialized data generated from the Weisman-Klemp experiment run on the Torus grid.
Add savepoints for datatest of both graupel microphysics and saturation adjustment since the same experiment is used for serializing the data in graupel and saturation adjustment calls.

Some additional fixes are:

add tracer state.
the computations in diagnose_pressure, diagnose_temperature, and diagnose_surface_pressure are changed accordingly as we now have moisture-related tracers.
Changes import statement in the above stencils and other stencils used in icon4py driver initialization.

OngChia · 2024-08-13T11:35:41Z

cscs-ci run default

OngChia · 2024-08-13T11:35:49Z

launch jenkins spack

OngChia · 2024-08-13T15:37:06Z

cscs-ci run default

halungge

I am a bit unsure on how to go on in relation with this PR concerning the interface there not the ADR. Should we try to make any "new" component into such an interface or is it better to keep the current "granule" style and then adapt them together with changing the driver?

I thought it would be good to take the first option, that is why I opened that PR in the first place, but I come to think that the second option is maybe better.

model/README.md

model/atmosphere/subgrid_scale_physics/microphysics/pyproject.toml

model/common/src/icon4py/model/common/states/tracer_state.py

model/atmosphere/subgrid_scale_physics/microphysics/tests/test_saturation_adjusment.py

halungge · 2024-08-14T11:18:58Z

...ics/src/icon4py/model/atmosphere/subgrid_scale_physics/microphysics/saturation_adjustment.py

+
+
+# TODO (Chia Rui): Refactor this class when there is consensus in gt4py team about the best way to express compile-time constants
+class ConvectTables(FrozenNamespace):


would that be a ConvectionTable? Or what does the Convect stand for?

It was named mo_convect_tables in ICON (convective adjustment table). But I just realized that it has became mo_lookup_tables_constants after I pull the latest branch... I now rename it to SaturatedPressureConstants in icon4py because, as far as I understand, it should only contain constants related to saturated pressure, which are used by saturation adjustment and microphysics components. Some constants such as rv and rd in SaturatedPressureConstants will be moved back to common/constants.py when the coding style of compile-time constants suggested by gt4py is fixed.

I think Nikki was working on that: GridTools/gt4py#1615 would that already fix the issue?

...ics/src/icon4py/model/atmosphere/subgrid_scale_physics/microphysics/saturation_adjustment.py

OngChia · 2024-08-15T15:02:43Z

cscs-ci run default

OngChia · 2024-08-16T06:36:51Z

Perhaps I will leave the saturaton_adjustment component as it is now in this PR. And I found that the input and output of graupel microphysics have just slightly more, but similar, arguments than saturation adjustment. The progress there is good now. I will open another PR after graupel merge to make these two components follow the interface protocol and more seriously think about integration into the driver. How is this plan?

OngChia · 2024-08-21T12:29:04Z

I have also added a few more diagnostic computations after the saturation adjustment.

halungge

As you have an entire new package the license update that we did (we switched from GPL to BSD 3 together with gt4py) is not completely smooth for you. I think you need to

merge the current main branch (that will update all the other packages)
update the pre-commit version in the pyproject of the microphysics package.
copy the LICENSE file from another package to your microphyscis package
re-run pre-commit
running pre-commit should hopefully update the license headers in all files inside your package.

halungge · 2024-08-23T07:48:47Z

model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/init_exner_pr.py

@@ -12,7 +12,7 @@



Is this a new stencil or is this one used in the blueline? If blueline, did you check for the dtype in ICON?

This stencil is called in subroutine perform_nh_stepping before the time loop starts in mo_nh_stepping.f90. It is only called once throughout the execution of the icon program. The stencil name has now become compute_exner_pert (previously it was called init_exner_pr). What it does is simply subtracts one variable from another variable, which should belong to a more generic stencil. Because it involves variables in DiagnosticStateHydro, I initially put it under solve_nonyhdro package.Now, I think that it should be put somewhere else. I will put it in driver/src/test_cases/utils.py for the time being, and I suppose in the future these kinds of generic stencils that only execute simple arithmetic operations will be created somewhere in common.
And thanks for asking me to check the dtype again. I made a mistake here. The correct type for exner_ref should be vp_float.

I also update its name to compute_perturbed_exner and add a docstring to the stencil citing where it comes from.

Liked the new name. Thank you. I always thought exner_pr stand for exner_pressure and was wondering why and what the difference was to the exner which is also exner pressure... Never guessed that _pr stands for "perturbation"... so much for the abbreviations.... 🙈

...ics/microphysics/src/icon4py/model/atmosphere/subgrid_scale_physics/microphysics/__init__.py

model/common/src/icon4py/model/common/constants.py

model/common/src/icon4py/model/common/states/diagnostic_state.py

model/atmosphere/subgrid_scale_physics/microphysics/.pre-commit-config.yaml

model/atmosphere/subgrid_scale_physics/microphysics/tests/__init__.py

halungge · 2024-08-23T09:05:30Z

...mmon/src/icon4py/model/common/interpolation/stencils/edge_2_cell_vector_rbf_interpolation.py

For my curiosity: what is actually the RBF part of this interpolation?

This stencil uses the C2E2C2E connectivity. The cell center value is computed from values located at the 9 edges of neighboring cells based on some RBF coefficients. I put a documentation (mostly copied from ICON) here for reference.

Ah so the interpolation coefficent that is used is generated by some RBF functions. But then would you agree that as the coefficients are arguments to the stencils (so any suitable interpolation coefficient, produced by whatever method) and the connectivity is just a connectivity, the stencil function itself independent of RBF?

We should maybe rename (not necessarily in this PR).

Highly agreed. I will rename it in my subsequent PR. let me think of a better name (maybe something like edge_2_cell_vector_interpolation_with_c2e2c2e_coef). I added a TODO there so that I won't forget.

model/common/tests/stencil_tests/test_diagnose_surface_pressure.py

OngChia · 2024-08-26T11:09:12Z

cscs-ci run default

OngChia · 2024-08-26T11:09:18Z

launch jenkins spack

OngChia · 2024-08-27T12:57:52Z

cscs-ci run default

OngChia · 2024-08-27T12:57:59Z

launch jenkins spack

OngChia · 2024-08-30T05:45:44Z

cscs-ci run default

halungge · 2024-09-03T08:51:23Z

Perhaps I will leave the saturaton_adjustment component as it is now in this PR. And I found that the input and output of graupel microphysics have just slightly more, but similar, arguments than saturation adjustment. The progress there is good now. I will open another PR after graupel merge to make these two components follow the interface protocol and more seriously think about integration into the driver. How is this plan?

That is fine for me.

halungge

I left some remarks, (more discussion points...)
For me is fine to leave the component interface to a separate PR.

github-actions · 2024-09-04T08:10:16Z

Mandatory Tests

Please make sure you run these tests via comment before you merge!

cscs-ci run default
launch jenkins spack

Optional Tests

To run benchmarks you can use:

cscs-ci run benchmark

To run tests and benchmarks with the DaCe backend you can use:

cscs-ci run dace

In case your change might affect downstream icon-exclaim, please consider running

launch jenkins icon

For more detailed information please look at CI in the EXCLAIM universe.

OngChia · 2024-09-04T08:10:21Z

cscs-ci run default

OngChia · 2024-09-04T08:10:28Z

launch jenkins spack

* empty microphysics package * merge from graupel branch. Saturation adjustment component in progress * saturation adjustment component completed * introduce type_alias in saturation adjustment * embedded graupel scheme tested ok * remove graupel from this branch * add documentation * precommit and clean up * remove redundant comment in test_saturation_adjustment * change physics to subgrid_scale_physics and removed redundant code * fix wrong rename * precommit and fix documentation * undo torus temporary fix and ammend satad documentation * update installation requirement text and readme in model package * fix model install requirement * review changes * add original name in ICON for constants * add more docstring * add diagnostic computation after saturation adjustment * fix potential bugs in diagnostic calculations and add in diagnostic part following satad call * new download link for wk experiment * split into two datatests for satad, fix bug, and add todo in diagnose_temperature * precommit * fix diagnostic stencil tests * move init_exner_pr * change import for diagnostic stencils and tests * precommit * fix license issue in microphysics package * add docstring for edge_2_cell_vector_rbf_interpolation * set virtual temp in driver test cases and fix import * add todo for WK experiemnt grid * rm init_exner_pr stencil test * add satad input output attributes, and common model attributes * use the new grid domain function in satad --------- Co-authored-by: Magdalena Luz <[email protected]>

halungge and others added 17 commits July 9, 2024 15:55

empty microphysics package

f8fde5d

Merge branch 'main' into microphysics_package

8b88e3d

merge from graupel branch. Saturation adjustment component in progress

828df27

saturation adjustment component completed

8daefcb

introduce type_alias in saturation adjustment

60f2f14

Merge branch 'main' into microphysics_package

251619e

embedded graupel scheme tested ok

786e66b

remove graupel from this branch

d643cf6

merge main

de5feb0

add documentation

9664b60

precommit and clean up

cc9dd21

remove redundant comment in test_saturation_adjustment

a2cde81

change physics to subgrid_scale_physics and removed redundant code

0055274

fix wrong rename

e605ab7

precommit and fix documentation

0f3be83

undo torus temporary fix and ammend satad documentation

b52a619

update installation requirement text and readme in model package

99f8aeb

OngChia requested a review from halungge August 13, 2024 11:35

OngChia added 2 commits August 13, 2024 17:35

Merge branch 'main' into microphysics_package

476c07e

fix model install requirement

9e49d53

halungge reviewed Aug 14, 2024

View reviewed changes

OngChia added 2 commits August 14, 2024 20:01

Merge branch 'main' into microphysics_package

4f10bc8

review changes

fe46f57

add original name in ICON for constants

6728941

add more docstring

3597c9f

OngChia requested a review from halungge August 21, 2024 12:07

halungge requested changes Aug 23, 2024

View reviewed changes

OngChia added 7 commits August 23, 2024 17:44

move init_exner_pr

615d798

change import for diagnostic stencils and tests

5591fe3

precommit

84e6007

fix license issue in microphysics package

b765132

add docstring for edge_2_cell_vector_rbf_interpolation

a6563ee

set virtual temp in driver test cases and fix import

f343e5a

add todo for WK experiemnt grid

ef1b7eb

OngChia requested a review from halungge August 26, 2024 11:09

rm init_exner_pr stencil test

7ba8f3c

add satad input output attributes, and common model attributes

e4c9c2d

OngChia added 2 commits August 30, 2024 11:22

Merge branch 'main' into microphysics_package

e40a3f3

Merge branch 'main' into microphysics_package

396a2c9

halungge approved these changes Sep 3, 2024

View reviewed changes

OngChia added 2 commits September 3, 2024 14:59

merge main

22afdfe

use the new grid domain function in satad

3dc536a

OngChia merged commit c78983a into main Sep 4, 2024
5 checks passed

halungge deleted the microphysics_package branch December 12, 2024 06:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Microphysics package with saturation adjustment #522

Microphysics package with saturation adjustment #522

OngChia commented Aug 13, 2024 •

edited

Loading

OngChia commented Aug 13, 2024

OngChia commented Aug 13, 2024

OngChia commented Aug 13, 2024

halungge left a comment

halungge Aug 14, 2024

OngChia Aug 15, 2024 •

edited

Loading

halungge Sep 3, 2024

OngChia commented Aug 15, 2024

OngChia commented Aug 16, 2024

OngChia commented Aug 21, 2024

halungge left a comment

halungge Aug 23, 2024

OngChia Aug 23, 2024 •

edited

Loading

OngChia Aug 23, 2024 •

edited

Loading

halungge Sep 3, 2024

halungge Aug 23, 2024

OngChia Aug 26, 2024

halungge Sep 3, 2024

OngChia Sep 3, 2024 •

edited

Loading

OngChia commented Aug 26, 2024

OngChia commented Aug 26, 2024

OngChia commented Aug 27, 2024

OngChia commented Aug 27, 2024

OngChia commented Aug 30, 2024

halungge commented Sep 3, 2024

halungge left a comment

github-actions bot commented Sep 4, 2024

OngChia commented Sep 4, 2024

OngChia commented Sep 4, 2024



		# TODO (Chia Rui): Refactor this class when there is consensus in gt4py team about the best way to express compile-time constants
		class ConvectTables(FrozenNamespace):

Microphysics package with saturation adjustment #522

Microphysics package with saturation adjustment #522

Conversation

OngChia commented Aug 13, 2024 • edited Loading

OngChia commented Aug 13, 2024

OngChia commented Aug 13, 2024

OngChia commented Aug 13, 2024

halungge left a comment

Choose a reason for hiding this comment

halungge Aug 14, 2024

Choose a reason for hiding this comment

OngChia Aug 15, 2024 • edited Loading

Choose a reason for hiding this comment

halungge Sep 3, 2024

Choose a reason for hiding this comment

OngChia commented Aug 15, 2024

OngChia commented Aug 16, 2024

OngChia commented Aug 21, 2024

halungge left a comment

Choose a reason for hiding this comment

halungge Aug 23, 2024

Choose a reason for hiding this comment

OngChia Aug 23, 2024 • edited Loading

Choose a reason for hiding this comment

OngChia Aug 23, 2024 • edited Loading

Choose a reason for hiding this comment

halungge Sep 3, 2024

Choose a reason for hiding this comment

halungge Aug 23, 2024

Choose a reason for hiding this comment

OngChia Aug 26, 2024

Choose a reason for hiding this comment

halungge Sep 3, 2024

Choose a reason for hiding this comment

OngChia Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

OngChia commented Aug 26, 2024

OngChia commented Aug 26, 2024

OngChia commented Aug 27, 2024

OngChia commented Aug 27, 2024

OngChia commented Aug 30, 2024

halungge commented Sep 3, 2024

halungge left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 4, 2024

OngChia commented Sep 4, 2024

OngChia commented Sep 4, 2024

OngChia commented Aug 13, 2024 •

edited

Loading

OngChia Aug 15, 2024 •

edited

Loading

OngChia Aug 23, 2024 •

edited

Loading

OngChia Aug 23, 2024 •

edited

Loading

OngChia Sep 3, 2024 •

edited

Loading