Adding Ionosphere Phase Estimation

[cmarshak]

Provides API functionality (through the associated CLI of the plugin) for:

1. ionosphere phase estimation (to be used for ionosphere correction)
2. ESD correction

Both of which are included in ISCE2 and requires modification of the topsApp.xml file (look at the example xml for all the features that can be included). Here is our template for this PR.

This pull request also packages the ionosphere layer within the GUWN product. Specifically, takes merged/topophase.ion.geo and puts into the netcdf group /science/grids/corrections/derived/ionosphere.

The ESD outputs are not preserved currently. This is because I am not aware how.

The current test for the PR is the following:

isce2_topsapp --reference-scenes S1A_IW_SLC__1SDV_20221002T151520_20221002T151543_045265_056931_E517 \
              --secondary-scenes S1A_IW_SLC__1SDV_20220908T151520_20220908T151542_044915_055D68_78EC \
              --estimate-ionosphere-delay True \
              --do-esd True \
              --esd-coherence-threshold .5 \
              > topsapp_img.out 2> topsapp_img.err

The above test is included in the readme.

Additional notes:

• I updated the inclusion of the ionosphere delay layer to use xarray (and rioxarray). It is much cleaner than writing netcdf attributes and coordinates directly. However, because there is a placeholder for/science/grids/corrections/derived/ionosphere typed as a string in the existing packaging template and workflow, I simply removed this from the packaging template json here. Reproducing the json excerpt below for posterity as the link is to the dev branch, which will presumably be updated if/when this PR is merged. There might be better ways of overwriting variables within xarray (specifically overwriting their type), but haven't looked into it. As a result, if ionosphere is turned off, the layer will be deleted (no placeholder).

                      "name" : "corrections",
                      "content" : [
                        {
                          "group" : [
                            {
                              "name" : "derived",
                              "content" : [
                                {
                                  "group" : [
                                    {
                                      "name" : "ionosphere",
                                      "content" : [
                                      {
                                          "dataset" : [
                                          {
                                            "name" : "ionosphere",
                                            "type" : "str",
                                            "description" : "placeholder dummy: [2D]"
                                          }]
                                      }]
                                    }]
                                }]
                            }

• The application of the ionosphere correction is turned off to allow for users to decide if/when/how to apply (see the xml template above).
• The test was chosen because there was clear "scalloping" (i.e. between-burst discontinuities in the unwrapped phase) in the existing GUNW (link).

[dbekaert]

@cmarshak just to double check if this is the unwrapped topophase or the wrapped variant.
Check if tehre is not a topophase.ion.unw or so

[cmarshak]

Let's look at the products together. I was going to just include this raster in the product.

[dbekaert]

We should check if we use all of these? I would only geocode the ones that gets used in the product. which would be fitlered coherence and filtered unwrapped phase corrected for DEM and topophase.

[cmarshak]

These are taken from here: https://github.com/aria-jpl/ariamh/blob/develop/interferogram/sentinel/topsApp.xml.tmpl#L32

[cmarshak]

Going to update example based on continued scalloping of the example initially given. More discussion is given here (by the insar experts): isce-framework/isce2#620

Here is new example given by @dbekaert:

isce2_topsapp --reference-scenes S1B_IW_SLC__1SDV_20171117T145926_20171117T145953_008323_00EBAB_AFB8 \
              --secondary-scenes S1A_IW_SLC__1SDV_20171111T150004_20171111T150032_019219_0208AF_EE89 \
              --estimate-ionosphere-delay True \
              --do-esd True \
              --esd-coherence-threshold .5 \
              > topsapp_img.out 2> topsapp_img.err

[cmarshak]

Using the SLCs that David recommended, I am attaching the browse image and the link to the resulting gunw.

Notes:

1. I am keeping the spatial_ref name that rioxarray uses for the ionosphere raster because it is too messy IMHO to rename the crs. This should not affect gdal/QGIS/panoply compatibility as you will see.
2. As noted above, this new branch will remove this group entirely if ionosphere is not requested.
3. The example has some faint "scalloping" (i.e. discontinuities between bursts). I do not know if ESD introduced this or the correction successfully removed most of the existing issues. Ultimately, it will be up to InSAR scientists to determine operational parameters for applications. Currently, the corrections, by default, will be turned off.

For futher discussion:

1. Should we increase the dataset version with these corrections added?

[dbekaert]

Some things to check:

• If the ionosphere is included the "version within the product/file" needs to be incremented to expose it to ARIA-tools that ionosphere exist. See the ALOS-2 scansar product we had prototyped before in HySDS or check ARIA-tools code-base for the version number to use. @sssangha might also recall. Migth be version 1c instead of 1b? This is important as ARIA-tools has the netcdf content dictionary mapping hard-coded based on internal version in the product. ARIA-tools will be able to support the ionosphere as we prototype it before too. e.g. run ARIA-tools extract on the iono layer to confirm it works.

Note that for tropo and offsets we will need to do this work still in ARIA-tools and we will also need to correct for the internal version. It could however be the same one as we have not used it beyond prototyping.

• You will also see by plotting up the ionospheric correction that it shows truncation/ different shape than the unwrapped phase. The correction should have the exact same spatial extent. So we should implement a masking step before packing it up. The meta-data to the layer looks ok and plots fine now in panoply.

In terms of future discussion item:
For now the external product version can be incremented minor for development purposes (product should not end up at the DAAC). However, i am contemplating to increment the major version but i want to get more change in tropo; iono, new frame, offsets. If we reprocess the archive we could push the older version data to the back in the collection.

to clarify

• Its unclear what the figure refers. Would be good to plot: 1) unwrapped layer from the product (this should not have the correction applied), 2) the ionospheric layer from the product, 3) a difference plot between both. My recollection on this example was slightly different. Likely it might be the filtering and 90 m product that smears things out a bit.

• From the text i understand you might have ESD applied. Large displacements (such as this EQ) can actually lead to a bias of the azimuth histogram. So would be good to include that plot. The best way to know if it made it worse is to run it one with and once without.

[dbekaert]

@ehavazli Can you have a look into this as well?

[ehavazli]

to clarify

• Its unclear what the figure refers. Would be good to plot: 1) unwrapped layer from the product (this should not have the correction applied), 2) the ionospheric layer from the product, 3) a difference plot between both. My recollection on this example was slightly different. Likely it might be the filtering and 90 m product that smears things out a bit.

1. Unwrapped phase layer from the product

2. Ionopshere layer from the product

3. Difference between ionosphere and unwrapped phase

The first and second plots are masked where the values are 0.

I would like to point out that the ionosphere layer is larger than the unwrapped phase and therefore would result in a larger image than the unwrapped phase if NOT masked before applying any the correction.

[EJFielding]

I would recommend not running the range split-spectrum ionospheric estimation and the ESD calculation on the same product as they both estimate azimuth shifts and will likely have a strong trade-off.
An earthquake coseismic interferogram can include substantial along-track deformation that will also affect the ESD estimation, as @dbekaert mentioned. This looks like the 2017 Mw 7.3 earthquake in Iran, and that earthquake has primarily east-west motion so it has less along-track deformation than many earthquakes. The 2019 Mw 7.1 Ridgecrest earthquake has much stronger along-track deformation.

[cmarshak]

Unfortunately, I want things to break more and to not be a blocker for a lot of parallel development. I am going to merge and include additional tests in the future.