Fix to Div-Vor calculation - and consequences

[rtodling]

This PR is a follow up to a bug found when comparing tropopause pressure GEOVALS between GSI and JEDI.

This has been reported among the SWELL geovals to be looked into:

GEOS-ESM/swell#241

and it got to become an involved discussion w/ NCEP (Emily) helping out to look at this in their system - (closed PR https://github.com/JCSDA-internal/fv3-jedi/pull/1208)

The initial discrepancies in tropprs GEOVALS looked like this:

Although at first the thought was that JEDI had a troubled code somewhere, the problem has been found to be related to how GMAO's interface to GSI calculates Vorticity and Divergence.

With a revised calculation of DIVO in GMAO's GSI, the geoval comparison now looks like this:

This was bound to happen at some point, i.e., all the GSI vs JEDI comparison would reveal an error in GSI - again, this is an error specific to GMAO's interface to GSI and not a bug in GSI per se.

The question is then how's this impact GSI's increments? Before answering this question it is good to know what role TROPPRS plays in GSI. This is how TROPPRS is used in GSI:

1. in the CRTM_INTERFACE treatment of IR radiance observations.
2. in the massaged saturation specific mixing ratio when derivatives are requested and Q and T are forced-decoupled. This plays a role in the rescaled analysis variable for humidity.
3. in the quality control of satellite wind observations - this controlled by a flag (qc_satwid) that is set to false in the GMAO's settings of GSI - actually, this flag (and it false settings) has been introduced by GMAO exactly because the results of applying this QC had been shown to be detrimental to GMAO's cycle - today, the speculation is that this was found to be so exactly because of the buggy DIVO.
4. And a much more key place where DIVO is used is in the flow-dependent feature of the background error variances in GSI.

Although the geoval comparisons above give an idea for how TROPPRS differ, here I show the actual full field difference between buggy (middle) fixed (top) and difference (bot):

As pointed out above, this difference in trop-prs is due to an error in the Div/Vor calculation. Here is a difference of the vorticity fields (x 1e3) at about 200 mb.

And just to rule out any speculations of the vor-div in the original GMAO interface being flipped upside down of something, here is a figure of vorticity at 1 mb: top: fix, middle: original; bot: diff.

And just for completion, at a level like 400 mb, this is the difference in the divergence calculation:

With the fix introduced here, the question then becomes: what's the impact on the analysis increments? The answer depends on the field examined. For example, the temperature and humidity fields don't seem to change much. The increments below are all from 3DFGAT.

Here is a T-increment at 200 mb:

And here is Q at 850 mb:

All panels above are difference with a the present (before fix) configuration of GMAO's GSI. The bottom panels exploit the Q_CSATWIND option that performs quality control on the satellite wind observations based on tropopause pressure.

What the two figs above show is that the fix amounts to minimal impact on T and Q.

Unfortunately, this is not the case for the the wind. Here is a figure, simulation to those above, but for U wind at 400 mb:

Whether the qc-satwind knob is used to quality control satellite wind observations or not, the impact of the fix on the winds is substantial. The consequence of this to a cycling experiment will have to be evaluated carefully.

Just for reference, I will also show here the difference in the initial Jo tables from a run using the present GMAO GSI code (before fix; left) and one using the DIVO fix discussed here:

Initial-Jo

Final-Jo

A substantially higher amount of radiance observations (IR) are tossed with the fix-divo calculation. The consequence of such seem to impact the winds more than anything else ...

The difference in the wind must be a consequence of the flow-dependent B errors... A test is has been produced, here is a figure:

The fig above shows again U wind at 400 mb as shown above, expect that the bottom panel deals with the no-Flow-Dependency option, that is, two cases: a default 3DFGAT was run w/o flow-dep the same case was run again, but now using the div-vor fix discussed here; the difference is what's shown. In order words, without the flow-dependence feature, the bug in the calculation of div-vor is negligible. Unfortunately, we do make use of the flow-dependence feature and the div-vor calculation has a substantial, non-trivial impact in the wind increments.

Now, all the tests above were done in a 3DFGAT configuration - sort of similar to GEOS-IT. In hybrid 4DEnVar, as in our x-experiments or FP, the impact of the bug in the increments should be smaller, at least in the core atmosphere, since we only give 50% of weight to the climatological covariance term.

I will do some testing in Hyb-4DEnVar ... but this PR should not wait for those tests to go in ...

here is the Hyb-4dEnVar test;

divofix.mp4

Comparing the move above w/ the corresponding fig for 3DFGAT confirms the expectation that in hyb-4d the effect of the bug would be weaker, if nothing else because Bclim gets only half the weight at these levels in 4D.

[github-actions]

Label error. Requires at least 1 of: 0 diff, 0 diff trivial, Non 0-diff, 0 diff structural, 0-diff trivial, Not 0-diff, 0-diff, automatic, 0-diff uncoupled. Found: bugfix

[github-actions]

Label error. Requires at least 1 of: 0 diff, 0 diff trivial, Non 0-diff, 0 diff structural, 0-diff trivial, Not 0-diff, 0-diff, automatic, 0-diff uncoupled. Found: bugfix

[github-actions]

Label error. Requires at least 1 of: 0 diff, 0 diff trivial, Non 0-diff, 0 diff structural, 0-diff trivial, Not 0-diff, 0-diff, automatic, 0-diff uncoupled. Found: bugfix

[github-actions]

Label error. Requires at least 1 of: 0 diff, 0 diff trivial, Non 0-diff, 0 diff structural, 0-diff trivial, Not 0-diff, 0-diff, automatic, 0-diff uncoupled. Found: bugfix

[gmao-jjin3]

@rtodling The changes look good to me. Thank you very much such a detailed analysis!