Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fixup geopotential<->height calculation (Fixes #1075) #1082

Merged
merged 1 commit into from
Jul 5, 2019

Conversation

dopplershift
Copy link
Member

Previous implementations was a naive implementation that directly translated formulas from source material. It included a lot of division and subtraction, resulting in catastrophic cancellation and severe loss of floating point precision. New formulation is just an algebraic reformulation that avoids these problems.

In fixing #1075, I went ahead and refactored height_to_geopotential as well to address similar problems with floating point precision.

@dopplershift dopplershift added Area: Calc Pertains to calculations Type: Bug Something is not working like it should labels Jul 3, 2019
@dopplershift dopplershift requested a review from zbruick July 3, 2019 20:28
@dopplershift
Copy link
Member Author

Borrowing from @sgdecker 's example in #1075, here's the change by this PR:

Before

geopotential_to_height:

Phi: 201590.000  z32: 20597.000  z64: 20596.956894  d: 0.0431  e: 20596.957031
Phi: 201591.000  z32: 20597.000  z64: 20597.059397  d: 0.0594  e: 20597.058594
Phi: 201592.000  z32: 20597.000  z64: 20597.161900  d: 0.1619  e: 20597.162109
Phi: 201593.000  z32: 20597.000  z64: 20597.264403  d: 0.2644  e: 20597.263672
Phi: 201594.000  z32: 20597.500  z64: 20597.366906  d: 0.1331  e: 20597.367188
Phi: 201595.000  z32: 20597.500  z64: 20597.469408  d: 0.0306  e: 20597.468750
Phi: 201596.000  z32: 20597.500  z64: 20597.571911  d: 0.0719  e: 20597.572266
Phi: 201597.000  z32: 20597.500  z64: 20597.674414  d: 0.1744  e: 20597.673828
Phi: 201598.000  z32: 20597.500  z64: 20597.776917  d: 0.2769  e: 20597.777344
Phi: 201599.000  z32: 20597.500  z64: 20597.879420  d: 0.3794  e: 20597.878906
RMSE: 0.6135

height_to_geopotential:

Z: 20597.000  phi32: 201590.719  phi64: 201590.420532  d: 0.2982  e: 201590.421875
Z: 20597.100  phi32: 201590.719  phi64: 201591.396115  d: 0.6774  e: 201591.390625
Z: 20597.199  phi32: 201590.719  phi64: 201592.371697  d: 1.6529  e: 201592.375000
Z: 20597.301  phi32: 201596.375  phi64: 201593.347280  d: 3.0277  e: 201593.343750
Z: 20597.400  phi32: 201596.375  phi64: 201594.322863  d: 2.0521  e: 201594.328125
Z: 20597.500  phi32: 201596.375  phi64: 201595.298446  d: 1.0766  e: 201595.296875
Z: 20597.600  phi32: 201596.375  phi64: 201596.274029  d: 0.1010  e: 201596.281250
Z: 20597.699  phi32: 201596.375  phi64: 201597.249611  d: 0.8746  e: 201597.250000
Z: 20597.801  phi32: 201602.047  phi64: 201598.225194  d: 3.8217  e: 201598.218750
Z: 20597.900  phi32: 201602.047  phi64: 201599.200777  d: 2.8461  e: 201599.203125
Z: 20598.000  phi32: 201602.047  phi64: 201600.176359  d: 1.8705  e: 201600.171875
RMSE: 6.6930

After

geopotential_to_height:

Phi: 201590.000  z32: 20596.957  z64: 20596.956894  d: 0.0001  e: 20596.957031
Phi: 201591.000  z32: 20597.059  z64: 20597.059397  d: 0.0008  e: 20597.058594
Phi: 201592.000  z32: 20597.162  z64: 20597.161900  d: 0.0002  e: 20597.162109
Phi: 201593.000  z32: 20597.266  z64: 20597.264403  d: 0.0012  e: 20597.263672
Phi: 201594.000  z32: 20597.365  z64: 20597.366906  d: 0.0017  e: 20597.367188
Phi: 201595.000  z32: 20597.469  z64: 20597.469408  d: 0.0007  e: 20597.468750
Phi: 201596.000  z32: 20597.570  z64: 20597.571911  d: 0.0016  e: 20597.572266
Phi: 201597.000  z32: 20597.674  z64: 20597.674414  d: 0.0006  e: 20597.673828
Phi: 201598.000  z32: 20597.777  z64: 20597.776917  d: 0.0004  e: 20597.777344
Phi: 201599.000  z32: 20597.881  z64: 20597.879420  d: 0.0014  e: 20597.878906
RMSE: 0.0033

height_to_geopotential:

Z: 20597.000  phi32: 201590.422  phi64: 201590.420532  d: 0.0013  e: 201590.421875
Z: 20597.100  phi32: 201591.391  phi64: 201591.396115  d: 0.0055  e: 201591.390625
Z: 20597.199  phi32: 201592.375  phi64: 201592.371697  d: 0.0033  e: 201592.375000
Z: 20597.301  phi32: 201593.344  phi64: 201593.347280  d: 0.0035  e: 201593.343750
Z: 20597.400  phi32: 201594.312  phi64: 201594.322863  d: 0.0104  e: 201594.328125
Z: 20597.500  phi32: 201595.297  phi64: 201595.298446  d: 0.0016  e: 201595.296875
Z: 20597.600  phi32: 201596.266  phi64: 201596.274029  d: 0.0084  e: 201596.281250
Z: 20597.699  phi32: 201597.250  phi64: 201597.249611  d: 0.0004  e: 201597.250000
Z: 20597.801  phi32: 201598.219  phi64: 201598.225194  d: 0.0064  e: 201598.218750
Z: 20597.900  phi32: 201599.203  phi64: 201599.200777  d: 0.0023  e: 201599.203125
Z: 20598.000  phi32: 201600.172  phi64: 201600.176359  d: 0.0045  e: 201600.171875
RMSE: 0.0174

zbruick
zbruick previously approved these changes Jul 3, 2019
Copy link
Contributor

@zbruick zbruick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple changes make a big difference - looks good!

Previous implementations was a naive implementation that directly
translated formulas from source material. It included a lot of division
and subtraction, resulting in catastrophic cancellation and severe loss
of floating point precision. New formulation is just an algebraic
reformulation that avoids these problems.
@dopplershift
Copy link
Member Author

Merging over skewT failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Calc Pertains to calculations Type: Bug Something is not working like it should
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants