Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GNATSS Results off by a large difference of >2cm! #193

Closed
lsetiawan opened this issue Dec 1, 2023 · 1 comment · Fixed by #194
Closed

GNATSS Results off by a large difference of >2cm! #193

lsetiawan opened this issue Dec 1, 2023 · 1 comment · Fixed by #194
Assignees
@lsetiawan
Labels
bug Something isn't working

Comments

@lsetiawan
Copy link
Member

Overview

John identified big differences in results b/w Python vs Fortran code. See below.

The results of the original fortran scripts are:


----------------------------------------------------------------------------------
               FINAL SOLUTION FOR17759 EPOCHS, 53277 MEASUREMENTS:
----------------------------------------------------------------------------------
After iteration  2, rms residual =     21.79 cm, error factor =     5.216
 09 TRANSPONDER (17759 MEASUREMENTS):
   x =  -2604416.4042 +/- 1.61196D-04 m   del_e =      0.1806 +/- 1.60999D-04 m
   y =  -3705738.2206 +/- 1.58216D-04 m   del_n =     -0.0268 +/- 1.62177D-04 m
   z =   4472870.9024 +/- 1.56888D-04 m   del_v =         0.2056 +/- 1.52998D-04 m
   Lat. =  44.832681118 deg., Long. = -125.099792616 deg., Hgt.msl = -1832.6164 m
 10 TRANSPONDER (17759 MEASUREMENTS):
   x =  -2606819.0424 +/- 1.61196D-04 m   del_e =      0.1807 +/- 1.60998D-04 m
   y =  -3705463.8707 +/- 1.58216D-04 m   del_n =     -0.0267 +/- 1.62178D-04 m
   z =   4471710.7685 +/- 1.56888D-04 m   del_v =         0.2055 +/- 1.52999D-04 m
   Lat. =  44.817929409 deg., Long. = -125.126647165 deg., Hgt.msl = -1829.4395 m
 11 TRANSPONDER (17759 MEASUREMENTS):
   x =  -2606247.3505 +/- 1.61196D-04 m   del_e =      0.1807 +/- 1.60997D-04 m
   y =  -3703528.8255 +/- 1.58216D-04 m   del_n =     -0.0268 +/- 1.62179D-04 m
   z =   4473632.0917 +/- 1.56888D-04 m   del_v =         0.2055 +/- 1.52998D-04 m
   Lat. =  44.842324959 deg., Long. = -125.134817994 deg., Hgt.msl = -1830.5545 m

The results of GNATSS are:

--- 17759 epochs, 53277 measurements ---
---- FINAL SOLUTION ----
NDP1-1
x = -2604416.4271 +/- 1.611955e-04 m del_e = 0.1805 +/- 1.609979e-04 m
y = -3705738.2529 +/- 1.582154e-04 m del_n = -0.0274 +/- 1.621764e-04 m
z = 4472870.941 +/- 1.568879e-04 m del_u = 0.2608 +/- 1.529981e-04 m
Lat. = 44.832681113201346 deg, Long. = -125.09979261690818, Hgt.msl = -1858.3711800229146 m
NDP1-2
x = -2606819.0652 +/- 1.611955e-04 m del_e = 0.1806 +/- 1.609969e-04 m
y = -3705463.9031 +/- 1.582154e-04 m del_n = -0.0273 +/- 1.621770e-04 m
z = 4471710.807 +/- 1.568879e-04 m del_u = 0.2608 +/- 1.529986e-04 m
Lat. = 44.817929404342145 deg, Long. = -125.12664716628078, Hgt.msl = -1855.1942329929761 m
NDP1-3
x = -2606247.3734 +/- 1.611955e-04 m del_e = 0.1806 +/- 1.609965e-04 m
y = -3703528.8578 +/- 1.582154e-04 m del_n = -0.0274 +/- 1.62178e-04 m
z = 4473632.1303 +/- 1.568879e-04 m del_u = 0.2607 +/- 1.529979e-04 m
Lat. = 44.84232495350724 deg, Long. = -125.1348179949484, Hgt.msl = -1856.3092628994827 m
------------------------

So the good news is that the del_e and del_n results are very close between the two runs, <1mm. However, the actual transponder coordinates differ by ~2-3 cm and the del_v estimate differs by ~6 cm.

While the del_e and del_n are the most important results, we should investigate the others.

Originally posted by @johnbdesanto in #175 (comment)
@lsetiawan lsetiawan added the bug Something isn't working label Dec 1, 2023
@lsetiawan
Copy link
Member Author

After some investigation it seems that the issue comes the differences in the harmonic mean inputs!

With the current algorithm using scipy's hmean function, the resulting harmonic mean is 1482.919 for all the transponders, while the Fortran code has harmonic mean value of 1482.944 this is a 0.025 m/s difference in sound speed. This threw off the computation as when I did a run with the values of harmonic mean fixed at 1482.944, I get the same exact result as the Fortran code:

Load sound speed profile data...
Computing harmonic mean...
lat=44.83268136 lon=-125.0997949 height=-1832.822 internal_delay=0.2 sv_mean=1482.944 pxp_id='NDP1-1' azimuth=3.06 elevation=41.67
lat=44.81792965 lon=-125.12664945 height=-1829.645 internal_delay=0.32 sv_mean=1482.944 pxp_id='NDP1-2' azimuth=-107.66 elevation=41.67
lat=44.8423252 lon=-125.13482028 height=-1830.76 internal_delay=0.44 sv_mean=1482.944 pxp_id='NDP1-3' azimuth=134.54 elevation=41.59
Finished computing harmonic mean
Load deletions data...
Load travel times...
Cleaning travel times data...
Load GPS data...
Cross referencing transmit, reply, and gps solutions...
Filtering out data outside of distance limit...
Preparing data inputs...
Perform solve...
--- 17759 epochs, 53277 measurements ---
After iteration: 1, rms residual = 41.57 cm, error factor = 9.953
NDP1-1
D_x = 5.304814e-02 m, Sigma(x) = 1.611978e-04 m
D_y = -2.385925e-01 m, Sigma(y) = 1.582168e-04 m
D_z = 1.258934e-01 m, Sigma(z) = 1.568894e-04 m
NDP1-2
D_x = 5.304814e-02 m, Sigma(x) = 1.611978e-04 m
D_y = -2.385925e-01 m, Sigma(y) = 1.582168e-04 m
D_z = 1.258934e-01 m, Sigma(z) = 1.568894e-04 m
NDP1-3
D_x = 5.304814e-02 m, Sigma(x) = 1.611978e-04 m
D_y = -2.385925e-01 m, Sigma(y) = 1.582168e-04 m
D_z = 1.258934e-01 m, Sigma(z) = 1.568894e-04 m

After iteration: 2, rms residual = 21.79 cm, error factor = 5.216
NDP1-1
D_x = -1.129193e-05 m, Sigma(x) = 1.611964e-04 m
D_y = 7.007267e-07 m, Sigma(y) = 1.582160e-04 m
D_z = 1.099388e-05 m, Sigma(z) = 1.568885e-04 m
NDP1-2
D_x = -1.129193e-05 m, Sigma(x) = 1.611964e-04 m
D_y = 7.007267e-07 m, Sigma(y) = 1.582160e-04 m
D_z = 1.099388e-05 m, Sigma(z) = 1.568885e-04 m
NDP1-3
D_x = -1.129193e-05 m, Sigma(x) = 1.611964e-04 m
D_y = 7.007267e-07 m, Sigma(y) = 1.582160e-04 m
D_z = 1.099388e-05 m, Sigma(z) = 1.568885e-04 m

---- FINAL SOLUTION ----
NDP1-1
x = -2604416.4042 +/- 1.611964e-04 m del_e = 0.1806 +/- 1.609988e-04 m
y = -3705738.2206 +/- 1.582160e-04 m del_n = -0.0268 +/- 1.621775e-04 m
z = 4472870.9024 +/- 1.568885e-04 m del_u = 0.2056 +/- 1.529982e-04 m
Lat. = 44.832681118391704 deg, Long. = -125.09979261554842, Hgt.msl = -1832.6164386987289 m
NDP1-2
x = -2606819.0423 +/- 1.611964e-04 m del_e = 0.1807 +/- 1.609978e-04 m
y = -3705463.8707 +/- 1.582160e-04 m del_n = -0.0267 +/- 1.621781e-04 m
z = 4471710.7684 +/- 1.568885e-04 m del_u = 0.2055 +/- 1.529986e-04 m
Lat. = 44.81792940940473 deg, Long. = -125.12664716515607, Hgt.msl = -1829.4394918491228 m
NDP1-3
x = -2606247.3505 +/- 1.611964e-04 m del_e = 0.1807 +/- 1.609975e-04 m
y = -3703528.8255 +/- 1.582160e-04 m del_n = -0.0268 +/- 1.621790e-04 m
z = 4473632.0917 +/- 1.568885e-04 m del_u = 0.2055 +/- 1.52998e-04 m
Lat. = 44.842324958781674 deg, Long. = -125.13481799389469, Hgt.msl = -1830.5545215193063 m
------------------------

@lsetiawan lsetiawan self-assigned this Dec 1, 2023
@lsetiawan lsetiawan mentioned this issue Dec 1, 2023
Merged
@lsetiawan lsetiawan linked a pull request Dec 1, 2023 that will close this issue
feat: Add numba harmonic mean implementation #194
Merged
lsetiawan added a commit that referenced this issue Dec 4, 2023
@lsetiawan @carlosgjs @pre-commit-ci
feat: Add numba harmonic mean implementation (#194)
a5537f8 
* feat: Add numba harmonic mean implementation

Added '_sv_harmon_mean' numba implementation for
calculating harmonic mean that uses calculations
used within the original fortran code to get same
harmonic values. Now there's a 'method' value for the
main 'sv_harmonic_mean' function that allows user
to switch back and forth b/w the two methods.

---

Ref #193

* test(harmonic_mean): Separate scipy and fortran aka numba hmean test

Separate out value testing for the two methods of computing
harmonic mean. The fortran values are obtained from
previous discussions at #66 (comment).

* test: Add test for _sv_harmon_mean

Add simple test for using the new '_sv_harmon_mean'
function on dummy arrays.

* test: update raised error to KeyError for missing col

* refactor: Add NotImplementedError for other methods

Added a raise NotImplementedError for any other
harmonic mean methods that is neither "scipy"
nor "numba".

* chore: Apply suggestions from code review

Co-authored-by: carlosgjs <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: carlosgjs <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lsetiawan lsetiawan

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Add numba harmonic mean implementation
1 participant
@lsetiawan